It is expected that a useful quantum computing (QC) system would require millions of qubits within one system. The state-of-the-arts QC systems contain hundreds of qubits, which is still several orders of magnitude away from the ultimate goal. To scale the QC system, namely, to increase the number of qubits and QC circuit layers, and also to improve the performance such as fidelity, robustness, etc., we need to develop scalable techniques including high-density qubit devices, ultra-low power qubit interfacing circuits, 3D interconnection and packaging etc. This article will review the recent reported device, circuit and system-level techniques that contribute to the QC system scaling and may ultimately lead to a fully functional quantum computer.