My penis is smaller as I get older

I am 55 years old and my penis seems to become an inny most of the time. When I’m aroused, it is its normal size but at other times it goes up into me. Is this normal? Is it my age?