A quarter of the way through the season, the Cleveland Browns are what they are – a 1-3 football team searching for answers to repair the cornerstone of their team, the defense.

After two crushing fourth-quarter losses to the Chargers and Raiders, coach Mike Pettine is leading the charge to flip the script and steer the season in the right direction.

"I can't sit here and say that it's not a source of frustration, it's not a source of disappointment," Pettine said. "We need to play better."

Pettine said on Monday football is about one million little things, but the Browns have clearly identified the Achilles' heel haunting the team: Allowing big plays on defense.

In consecutive losses, both San Diego and Oakland shredded Cleveland's defense for a combined six plays of 40 yards or more. With tonight's Monday Night Football game still pending, the Browns rank last in the league allowing 406.2 yards per game.

For a unit littered with Pro Bowlers, the start of the season has been humbling.

"That whole room is upset," Pettine said. "It's prideful and its players and coaches alike."

Allowing the big plays would be less problematic if the unit was producing turnovers, but it hasn't. A year after the Browns finished second in the league in with 21 interceptions, Marcus Mariota, Philip Rivers and Derek Carr didn't throw picks in consecutive games. The Browns have one interception so far in 2015, and Tashaun Gipson ended up fumbling that pick back in Week 1 against the Jets. Cleveland is tied for 16th in the league in takeaways.

The most exasperating part of Cleveland's 1-3 start is that if the defense was playing up to its potential, there's a chance this team could be sitting at 3-1. Johnny Manziel delivered one win and Josh McCown was on the doorstep for two others. The offense has heavily surpassed the early season expectations set by those outside of the building.  

"It's a team game," McCown said in the locker room Monday. "All we do is on either side is everybody looks at what they could have done to help us be better and play better and make those corrections and move forward."

So, that's what's next: How can Cleveland channel its 2014 performance on defense, a unit that propelled the team to a 7-4 record late in November? How will this team come up with answers to win football games in the fourth quarter?

Pettine was adamant there will be no wholesale changes on defense and his faith in coordinator Jim O'Neil has remained unwavering. The coaching staff reviews each negative play on defense, trying to categorize patterns.

"Was it because it was a technical error that we made? Pettine said. "Was it a scheme mistake? Was it personnel?"

Cornerback Tramon Williams says the Browns can't get angry at themselves; they must stay composed.  Pettine didn't question the team's veteran leadership, either.

"We have to be that team that if a mistake is made, we learn from it, we lock it away and we don't make it again," Pettine said.

The resounding response from the locker room is that this team is close to stringing wins together. It's a bottom-line business, and the Browns lost, but Cleveland left it all on the field Sunday in San Diego.

"It is not rocket science," linebacker Christian Kirksey said. "It is not something that we have to search to find. We have it. We just have to make sure we execute it on Sunday."

Cleveland's shift on offense, which centered on short passing – 42 passes compared to just 21 runs – had San Diego's defense on the ropes. And it's not all gloom and doom with the defense, either. The Browns run defense stifled hyped rookie Melvin Gordon for just 38 yards on 12 carries. Coming off a 12 catch, 133-yard takedown on the Vikings, receiver Keenan Allen was held mostly in check with four catches for 72 yards, even without Joe Haden. Pettine noted the tackling was much improved after it became a large emphasis in practice after the Oakland game.

"The encouraging thing is that we made strides in some areas," Pettine said. "Then there are areas where we know we have to get things cleaned up and we know we are capable of doing it."