Going to New York by ferry, a luxury almost no one can afford

Manhattan could be closer than it is today. If you live in Hoboken, and you have to get to the City using public transports, the cheapest and faster ways so far are either subway or buses. Actually there would be a third one, but almost nobody seems to consider it. Why? Because it's senselessly expensive.
A one-way ride by ferry from Hoboken to New York Port Authority costs 9 dollars, more than three rides by Path or two by NJ Transit. No surprise ferries run almost empty all the times. Yet, it would be a faster - and even more pleasant - commute than by bus: who would like to be stuck in a traffic jam in the tunnel at rush hours rather than watching Manhattan approaching from the deck of the boat, the wind on the face? Nine bucks per ride is a good point, although.
What happens on the other side of Manhattan? It's a totally different story. Ferries sailing East River to Brooklyn are cheap (a single ride costs as much as a regular MTA ticket) and crowded. The ones to Staten Island are free.
It's understandable that Interstate rides have to cost a little more than an ordinary ticket. So why don't charge the Interstate waterway 3.50 dollars as a bus ride? Besides, how much do ferries rides cost now to the community and to the operator, given that so few people use them? We the Hobokenites would love so much to go to New York by the water, if only was more affordable...