Bay came through after he was robbed of a game-winning homer in the 11, when David Murphy leaped above the fence in left-center field to catch Bay's drive.
Raul Ibanez tied it at 3 earlier in the 11th with a home run. A.J. Pierzynski had given Texas a 3-2 lead in the top half with a pinch-hit RBI single off Oliver Perez.
Yoervis Medina (1-0) worked out a jam in the top of the 13th.
Kendrys Morales doubled and advanced to third on a groundout by Carlos Triunfel. Bay then singled off Rangers left-hander Michael Kirkman (0-1) to drive in Triunfel to earn the victory.
Jurickson Profar hit a leadoff home run off Mariners starter Hisashi Iwakuma. Texas scored again in the second after hits by Nelson Cruz and Mitch Moreland put runners in scoring positions. Leonys Martin drove in Cruz on a sacrifice fly to give the Rangers a 2-0 lead.
The Mariners left a pair of runners stranded in both the fourth and fifth innings before finally breaking through against Nick Tepesch in the sixth. Kyle Seager doubled down the right field line and Morales homered to knot the game at 2.
Tepesch went 5 2-3 innings for Texas, allowing just five hits and two runs. He walked one and struck out six.
The Rangers fumbled an opportunity to possibly retake the lead in the seventh inning. After a leadoff double from Adrian Beltre, Cruz struck out swinging at a ball in the dirt. Beltre was caught between second and third base as the ball bounced away from Mariners catcher Jesus Sucre. Cruz tried to take first base on the play but collided with Sucre and was ruled out. Sucre then ran down Beltre and he was called out for running out of the baseline.
Pierzynski delivered a pinch-hit RBI single in the 11th inning off Seattle Mariners reliever Oliver Perez to give the Rangers a 3-2 lead.
Ibanez homered off Rangers closer Joe Nathan into the right field seats to even the game at 3-all in the bottom half. It was the first blown save of the year for Nathan. He had been 9 for 9 in save opportunities this season.
Iwakuma had another strong outing for Seattle. He allowed two runs and two hits over eight innings. He struck out eight.
NOTES: Profar is the youngest player to hit a leadoff homerun (20 years, 95 days) since Lou Klimchock (18 years, 348 days) of the Kansas City Athletics homered on Sept. 28, 1958, against the Chicago White Sox. ... Nathan had his streak of 36 consecutive saves on the road snapped with Ibanez's tying home run in the 11th inning. Nathan had converted all 31 save opportunities on the road since joining the Rangers last year. ... Mariners 1B Justin Smoak is day to day after straining an oblique Saturday night. Morales started in his place. Michael Morse also is expected to see time at first base in Smoak's absence.