The theoretical model of RIKEN mesoscopic device in our study is one kind of the two-dimensional Sinai billiards, which is an ideal model to investigate the chaotic and fractal behaviors in particle escape curves. In the analysis of the escape curves, we use two methods:qualitative comparison and quantitative calculation of the fractal dimensions. Then we obtain the influence of the distribution of chaotic areas caused by the opening width, cavity length, corner positions, arc radius, etc. In the results, we find the fractal self-similar structure of the escape curves in which the similarity is very good, and they display the chaotic property of the transmission in the RIKEN device. Moreover, we use the eye-style structure analysis and the comparation between similar ratios to investagate the fractal self-similar structure.