You can do that when walking around the sandstone walls of Castle Clinton at Battery Park. Considering the size and power of the nation today, it is interesting to think of a young America in more precarious times. Castle Clinton was built between 1808-1811 on the shores of Manhattan in effort to help protest a young and vulnerable America against British invasion.