My university has a "Stellar Physics" course with only first-year physics courses as a prerequisite. Classification of stars, their composition & lifecycle, planetary formation & dynamics, etc. But for more intense stuff? Not a physicist but on the math side at least you're going to need to be confident in vector calculus, able to analyze and solve ordinary & partial differential equations, maybe some abstract algebra. These will all be needed to reason with advanced physics equations. Only guessing on the physics side but relativity, condensed matter and thermodynamics will all be involved.
>>683 echo what this lainon said. Astronomy without a high level of math and physics ability will just land you in the unemployment office--if this is just a hobby, there may come a point where yuo are tired of just knowing the names and explanations of things.