Here's a map of were we've been so far.  The part where it looks like we drove out into Tokyo Bay was when we used the Tokyo Aqua Line tunnel. It's about a 15km stretch between the Tokyo side and the Chiba side.  2/3 is a tunnel under the bay, and 1/3 is a bridge.  Where the tunnel and bridge meet they built a huge rest stop with shops and restaurants.