Nonlocal quantum entanglement is a fundamental resource for future quantum networks. However, the efficiency of generating nonlocal entanglement between distant nodes is severely limited by the exponential loss incurred when locally generated entangled states are distributed through lossy quantum channels. This limitation becomes more pronounced in practical scenarios requiring the simultaneous distribution of multiple entangled pairs. While classical multiplexing approaches, such as spatial, temporal, and frequency multiplexing, can increase the nonlocal entanglement generation rate, they do not improve the single-shot transmission efficiency. In contrast, quantum multiplexing, enabled by high-dimensional encoding of single photons, allows the parallel generation of multiple nonlocal entangled pairs in a single transmission round, thereby enhancing the overall efficiency of nonlocal entanglement generation. Quantum multiplexing thus offers a promising route toward scalable quantum networks. This review introduces the mechanisms for generating nonlocal entanglement using quantum multiplexing. It focuses on two main approaches using high-dimensional single-photon encoding and high-dimensional biphoton entanglement distribution, respectively. It then examines how quantum multiplexing can accelerate the generation of nonlocal quantum logical entanglement. Finally, it briefly explores the potential of quantum multiplexing for building large-scale quantum networks.