Don't you think it's all pretty arbitrary?

Because we see seven colors in a rainbow doesn't mean there *are* seven colors. A person from a different culture may very well see only four distinct colors in the rainbow, while others (say an interior decorator or a painter) may see many more.

The attribution of colors to chakras may be arbitrary. It may just be a way of categorizing and understanding things, not a truth in and of itself.

Higher and lower frequencies of color and vibration make sense as correlaries, but, really, "up there" may no be so heavenly and "down there" may not be so earthy or base.

Up and down are both based on frames of reference that change. For instance, if you're upside down, then the relationship of colors of the rainbow to the chakras matches.

And, also, in experiments with light and refraction through lenses (and that is, I think, what a rainbow is) things often get reversed ... so, it very well may be that purple is indeed "up" and red is "down."