Earths early atmosphere was largely made up of large amounts of carbon dioxide, little to no oxygen and other gasses such as methane and ammonia. There was also water vapour which was released by volcanoes when they erupted.
When the water vapour released by volcanoes condensed as the earth cooled, it formed the oceans. Nitrogen built up in the atmosphere (as it is unreactive) when it was released from volcanoes also.
For millions of years, the gasses in our atmosphere have been pretty stable. The gasses had to become stable from the original atmosphere first, which was largely carbon dioxide.
As plants developed they photosynthesized. Oxygen is a by-product of photosynthesis which led to an increase of photosynthesis.
Carbon dioxide + water --> Glucose + oxygen
As we can see in the photosynthesis equation, carbon dioxide is used and oxygen is released, leading to an increase of oxygen and a decrease in carbon dioxide.
Carbon dioxide decrease
As we can see above, plants use carbon dioxide which led to a decrease in carbon dioxide in the atmosphere. Aswell as this it also decreased because it is very soluble in water and so with the newly formed oceans, there was lots of water which would absorb the carbon dioxide, therefor reducing the amount of carbon dioxide in the atmosphere. As carbon dioxide dissolved it formed carbonate compounds which would go on to precipitate into sedimentary rocks, eg limestone.