We wanted to walk High Bridge, the oldest bridge in Manhattan, even though it has been closed to the public for many years. An old abandoned staircase leads down to the bridge from High Bridge Park.
But hopping the razor wire fence proved to be too daunting- at least from the Manhattan side. I've heard there is a rope you can crawl up on the Bronx side- maybe I'll try it some time. I took the photo below through a gap in the fence.
High Bridge was built in the 1840's to carry water from the Croton Reservoir into Manhattan. The water was then pumped up into a tower, which still rises above the park.