More Agency Report Comments

Yesterday, we finally obtained a full copy of the Public Redacted Version of Boeing's Comments on the Agency Report which we have now posted in two parts. Part 1 pages 1-92 and Part 2 pages 93-191 (Note: Each are big 6 MB files but they are in streaming text format so viewing should not should not take too long.)

Also yesterday, the KC-30 team made available their Agency Report Comments. As is their recurring theme, the KC-30 submission came in much bigger and heavier; totaling 370 pages of comments. (Note: This is a huge 13.2 MB PDF file).

Here is how Boeing summarizes their comments:

The Air Force rests its submission that Boeing's protest should be denied on the claim that Boeing merely disagrees with its subjective professional judgment. On issue after issue after issue, the Air Force dismisses the concerns Boeing has raised by intoning the mantra that its position reflects "the evaluators' professional and subjective judgment." It is certainly true that GAO will defer to the reasoned exercise of the agency's professional judgment. But when the claimed exercise of professional judgment is contrary to the plain terms of the Request for Proposals (RFP), when it is exercised disparately, when it is patently unreasonable, the Air Force's judgment is not entitled to deference. And when, on issue after issue after issue central to the source selection decision, the Air Force's disregard for the plain terms of the RFP, its disparate treatment of the offerors, and its patently unreasonable judgments consistently redound to the benefit of one offeror and to the detriment of the other, the protest must be sustained.

Here is how the KC-30 team concludes their comments:

Boeing has made many different claims in its multiple protest filings. The vast majority share a common unstated premise - echoed in its PR campaign - that the Air Force was prejudiced against Boeing and intent on treating it unfairly. An examination of the facts surrounding Boeing's grounds of protest demonstrate just the opposite. The Air Force conducted a comprehensive evaluation which treated both offerors fairly. If anything, it went out of its way to give Boeing the benefit of the doubt. On a few minor points the Air Force has made corrections after considering the Boeing protest arguments. Perfection, while an admirable goal, is rarely achieved in human affairs, and particularly not in something as complex as the evaluation which the Air Force performed here. But those small corrections are inconsequential and in no way alter the conclusion that the SSA reasonably determined that Northrop Grumman submitted the more advantageous proposal. Certainly, nothing in the record supports the exaggerated claim of deliberate manipulation and bias throughout the Air Force on which Boeing's protest depends.

After reading portions of both sets of comments, we can say that good arguments are made on each side. It seems almost impossible for an outsider to say beyond the life cycle cost errors we mentioned previously what the GAO's will decide.

Are the admitted errors minor and inconsequential as both the Air Force and the KC-30 team claim? It difficult to tell without seeing the Agency Report and the full comments. Also, much of the protest rests on the technical merits of each proposed aircraft; not our strong suit.

A key point will be if the GAO believes the metrics in many of the disputed areas were set by rigorous analysis or instead by a what Boeing claims is "flawed analysis" and "unreasonable judgement".

In Boeing's favor was a GAO Report released on 6 March 2007, after the final KC-X RFP, entitled Defense Acquisitions: Air Force Decision to Include a Passenger and Cargo Capability in Its Replacement Refueling Aircraft Was Made without Required Analyses. As the title makes plainly clear the GAO at that time did not believe that the DoD had established in any concrete way that it had done the proper analysis:

Mandatory Air Force implementing guidance governing the capabilities-based planning analyses discussed above states that capabilities-based planning employs an analysis process that identifies, assesses, and prioritizes needed military capabilities. These four analyses did not identify a passenger and cargo capability gap, did not establish that such a capability would represent a redundancy, and did not assess the risk of not acquiring such a capability. Without sound analyses, the Air Force may be at risk of spending several billion dollars unnecessarily for a capability that may not be needed to meet a gap or shortfall.

Military Decision Makers Approved the Capability with Neither an Identified Need nor Risk Assessment

DoD strenuously disagreed with the GAO's assessment then, but the question remains: Did the Air Force prove that their grading criteria had merit, and if so did they set that criteria prior to the RFP, and did they carry out the grading rules the criteria set evenly?

In Boeing's comments they clearly state their position on this:

The source selection decision repeatedly violated the terms of the Solicitation in material respects. These legal errors permit GAO to sustain the protest without a hearing, but they do not begin to capture the full measure of the Air Force's flawed analysis.

Here in DC, Tanker War Blog is not alone in thinking the Air Force made mistakes, but until the GAO decision is released, no one can know for sure the magnitude of those errors.