CeRh₂As₂, as a recently discovered Ce-based 122-type heavy-fermion superconductor, has attracted much attention due to its non-Fermi-liquid behavior and two-phase superconductivity. The tetragonal crystal structure of CeRh₂As₂ maintains global centrosymmetry, which makes even-parity and odd-parity superconducting states different rather than mixed. The Ce site exhibits local inversion symmetry breaking, which results in staggered Rashba spin-orbit coupling. This may lead to the c axis field-induced transition between two superconducting phases and high critical field. Given the novel physics in CeRh₂As₂, including a possible quantum critical point and a spin-fluctuation-mediated superconducting pairing mechanism, the ultra-low-temperature electrical and thermal transport properties of CeRh₂As₂ under various magnetic fields are investigated in this work. The zero-field resistivity reveals a superconducting transition at the critical temperature T_c = 0.34 K. At a magnetic field of 1 T, a minimum resistivity appears near T₀$ \approx $0.42 K, which may be due to partial gap opening caused by Fermi surface nesting, indicating that the system enters into a magnetically ordered state, which is not observed in zero field. In the temperature range from T₀ to 2 K, the system exhibits non-Fermi-liquid behavior $ \rho\sim{{T}}^{0.44} $, indicating proximity to a quantum critical point. The superconducting transition is fully suppressed at 7 T, with resistivity recovering Fermi-liquid behavior at low temperature. No significant anomaly is observed in the zero-field thermal conductivity of CeRh₂As₂ near T_c. This absence of anomaly may be attributed to the high residual resistivity of the sample, and the reduction in carrier density during the superconducting transition and the T₀ phase transition. It requires optimizing single crystal growth to reduce the effects of lattice defects or chemical disorder on thermal transport. Upon applying magnetic field, the thermal conductivity curve exhibits a small upward shift relative to its zero-field curve. At 0.15 K, thermal conductivity rises with the increase of magnetic field and is saturated at higher fields (above 5 T). In the normal state at 7 T, it is found that the electrical resistivity and thermal conductivity satisfy the Wiedemann-Franz law, indicating that the charge transfer and the heat transport are governed by the same quasiparticles, which is consistent with the Fermi-liquid behavior observed in resistivity under this field.