Ryan Braun and Prince Fielder hit back-to-back homers in the third against Karstens (2-4), and despite the Pirates putting runners on base in all but one inning against Milwaukee's Dave Bush (4-6), runs remained hard to come by.
Karstens was particularly irked with his high proportion of walks, a career-high six (one intentional) in five innings. The right-hander entered the game with just 16 walks in nearly 73 innings.
"The two home runs are going to happen, but the run in the first and the run in the fifth, I feel like it can't happen," Karstens said. "If they hit their way on, I tip my hat. But I walked those guys, put them in good situations to score runs, and they took advantage of it."
Braun walked and scored on a two-out single in the opening inning, and the intentional walk to Fielder in the fifth helped load the bases for Craig Counsell's sacrifice fly, which gave the Brewers a 4-0 lead and ultimately provided the winning tally.
"I had bad location all night," Karstens said. "It's just a sour taste in my mouth when I go out there and walk five guys. That's 20 percent of the guys I've walked all year.
"The reason I had success early because I was my own pitching coach and made adjustments out there after pitches that weren't good. Tonight, I didn't do that."
The blasts by the Brewers' dynamic duo marked the ninth time in 11 starts this season that Karstens has allowed at least one long ball. Of his 33 runs allowed as a starter, 17 have scored on home runs. He finished with four runs allowed on five hits.
"He was out of sync a little bit," Pirates manager John Russell said. "He made some pitches that got him behind in the count, and when you do that against a team like this, they're going to take advantage of it. It could have been worse. He could have really let it cave in on him, but he didn't. He got through most of it, but the home runs really hurt him."
Sean Gallagher's debut in a Pirates uniform proved shaky when he loaded the bases with one out in the sixth, but the right-hander struck out Braun, and Javier Lopez finished the scoreless frame by striking out Fielder.
D.J. Carrasco worked a clean seventh and induced a popout from Braun to end a dicey eighth. On Friday, Braun delivered a game-winning double in the 10th off Evan Meek to complete a late comeback. According to the Elias Sports Bureau, the Pirates were the only remaining team in the National League that had not lost a game after holding a lead in the ninth or later.
This time, Carrasco retired Braun with two runners on base, but Braun still finished with three runs scored and his 33rd multi-hit game of the year.
Jones homered leading off the sixth to put the Pirates on the board and cut Milwaukee's lead to 4-1.
"I think we're playing decent baseball -- it's just timing," Jones said. "There's really not much to say or explain. We hit the ball pretty well overall, and we're pitching well. In close games, it's the timely hitting.
"We do it at home, so there's no reason why we can't do it on the road."
Bush worked into the seventh for Milwaukee, allowing two earned runs on six hits.
"I thought Bush's curveball was real good tonight," Brewers manager Ken Macha said. "He got a bunch of their left-handed hitters -- [Ryan] Church, [Pedro] Alvarez, [Neil] Walker -- chasing the curveball in the dirt with two strikes. Very effective with the change of speeds on that."
Andrew McCutchen hit a sacrifice fly in the seventh. Milwaukee's John Axford pinned down his 10th save, but not before Doumit crushed an offering to deep right.
Pittsburgh's five-game losing streak has coincided with a streak of games in which it has allowed a first-inning run, and the Pirates have now given up 15 home runs over the past seven games.
"Guys need to relax a little bit more," Russell said. "We're seeing signs of improvement, but it's just not good enough. At some point, we'll get over the hump, but it's been a battle for us. Some guys are doing some better things, but we're not to the level where we can get it done and get more runs across."