InfoMullet: Boundary & Factor Analysis on WhistleBlower Allegations
TLDRUpFront: Given the importance of the impeachment inquiry, I’m launching a live tracker of the accumulating evidence and evaluation of, for or against a charge, using a specific analytical method called “Boundary and Factor Analysis.” This post is a description of the method. I hope people find this approach informative and educational as one way in which to approach complex and controversial topics in their own lives. This is just one alternative way to approach critical thinking in these kinds of situations. But we often don’t spend enough time discussing the modes of critical thinking in lieu of focusing on the output.
FullContext in the Back:
The method I’m using is what I call “boundary & factor analysis” which is a super-simplified notional form of heuristic factor analysis. And heuristic factor analysis is actually the cousin-in-ambiguity to component factor analysis, or absolute factor analysis. To be clear this is a method and not a mathematical exercise. But it’s one way to think, sort, and organize complex information when having to arrive at crucial decision making in environments of high ambiguity and controversy. Though I’m going to be using it in a qualitative way without the math, my hope is the process and method remains rigorous and builds confidence in the findings, while also being transparent and subject to challenge.
The standard of evidence I’ll be using in this exercise is “substantial”, “clear & convincing”, and “credible.” These are specific objective legal standards in the US court system that come from three different types of cases. I’ll define them further below, as well as why I selected these.
Process of Boundary & Factor Analysis
- Establish initial set of independent and contradictory hypotheses, each hypothesis being a smaller element within the larger narrative of a charge.
- Create a Boundary within which all potential hypothesis lie.
- Review evidence that lies within the boundary collecting more as needed, ensuring that it is credible.
- Perform Factor Analysis
a) Distill observations into factors
b) Weight factors (e.g. evidence credibility, prompted v unprompted revelations, contemporary documentation vs. delayed vs. contemporaneously documentation vs. reported vs. reporting with apriori knowledge etc.)
c) Tag factors as confirmatory, refuting, confounding, or irrelevant to a given hypothesis within the set.
d) Ensure confirmatory or refuting factors are convincing.
e) Assess whether any hypotheses should be rejected for lack of confirmatory factors.
f) Compare mutually exclusive hypotheses: Does the notional sum of the weighted factors (confirming – refuting – confounding) of one hypothesis substantially exceed the notional sum of the other? If so reject the less substantial. If not, retain both and seek additional factors from witnesses until one is resolved in favor of the other.
g) Have confounding factors generated new mutually exclusive hypothesis which must be resolved? - Iterate by adjusting the boundary. Narrowing the boundary as rejected hypotheses are removed or expanding it to include newly arisen mutually exclusive hypotheses.
- Repeat as necessary until the boundary only contains a set of hypotheses that are not mutually exclusive and clear & convincing.
SOURCING AND CREDIBILITY
Credibility will be assessed on a falsifiable scale that increases in credibility from alleged, reported, corroborated, confirmed, and admitted.
Allegations are simply that, statements by people alleging something has been done. These may come from sources that don’t meet the credibility threshold listed below.
Reporting meets a credible threshold when at least three sources are reporting the same information. These sources must all lie within the “most reliable news” sector (score of 40+ on Quality and between -20 and +20 on the political bias of AdFontes media and no further than Left-Center or Right-Center on Media Bias Fact Check. (All sources are in first comment.) For credible levels below, the same standard will be used as reporting for accepting that corroboration has occurred. If a news source isn’t rated in AdFontes, but receives a “Least Biased” rating from Media Bias Fact Check, it is allowed but won’t be the only source relied upon.
Corroboration is when corroboration has been obtained between principle actors witnessing the events first hand.
Confirmed is the release of a government document, finding, transaction statement or report. These are weighted by whether they were contemporaneously reported at the time, or after the fact, and the partisan bias of the organization releasing it. For example, a Pentagon transactional memo, required by law for certain kinds of military transactions, will be of higher confirming credibility to its related factors than a finding by a Congressional committee that split the vote along partisan lines for the finding.
Admitted factors are those that the principal actors involved themselves have admitted too, or caused to be admitted, either by direct statement in the media and/or by approval or release of information to the public. In these cases, sources may be used that are not deemed credible for other sources but are the copyright owners of video within which the admission was made.
Because I’m using notional weighting, the number of sources I use won’t be consistent for each factor. For example, if the Factor is about a public law, and I cite a Confirmed source which is the public law, that’s good enough. But if the Confirmed source is a Congressional Committee finding that had partisan splits, I’ll seek out more evidence and sources. Likewise, admissions of wrongdoing will be weighted higher as a source than admissions of “right-doing”, which will require more sourcing to verify whether the admission is self-serving or not.
EXAMPLE OF PROCESS
By way of example let’s consider a “wicked mess” case of unraveling the mystery of “who ate the KFC that was delivered.”
Initial Boundary:
The house within which the KFC was alleged to be located and all parties (Brad and Tim) who may have consumed it or have knowledge of its consumption.
ROUND ONE
Boundary of All Active Hypotheses at Start of Iteration:
a) Tim ate the KFC in the house.
b) There was no KFC in the house.
c) Tim was never in the house.
d) Brad was never in the house.
e) Brad ate the KFC in the house.
f) There was KFC in the house.
Rejected Hypotheses: None
Accepted Hypotheses: None
Hypotheses b & c are mutually exclusive to hypothesis a. Hypotheses b & d are mutually exclusive to e, and hypothesis a and e are both mutually exclusive as are b & f. If this sounds a bit like “Clue” it’s not far off.
We now collect and evaluate factors of evidence through iteration. Let’s say for the first iteration Brad alleges Tim ate the KFC, that’d be a confirmatory factor for hypothesis a & f and refuting b, c, & e. Let’s say Tim alleges there never was any KFC in the house, which would be refuting of a & f, and confirmatory to b. Let’s say Tim also alleges Allison was in the house. This would be a confounding factor because now we have a third actor who may have eaten the KFC, even if Tim didn’t say so.
After the first iteration, our boundary of hypotheses looks like this:
Boundary of All Active Hypotheses at End of Iteration:
a) Tim ate the KFC in the house. (Alleged)
b) There was no KFC in the house. (Alleged)
c) Tim was never in the house.
d) Brad was never in the house.
e) Brad ate the KFC in the house. (Alleged)
f) There was KFC in the house. (Alleged)
g) Allison ate the KFC in the house. (Alleged)
Rejected Hypotheses: None
Accepted Hypotheses: None
There is no hypothesis with sufficient credibility to either accept or reject. We iterate again. This time we find that three credible news sources have Alison at a different location, which creates a confounding and mutually exclusive hypothesis to g. We also find a receipt for UberEats documenting that KFC was delivered to the house at the time/date in question.
ROUND TWO
Boundary of All Active Hypotheses at Start of Iteration:
a) Tim ate the KFC in the house. (Alleged)
b) There was no KFC in the house. (Alleged)
c) Tim was never in the house.
d) Brad was never in the house.
e) Brad ate the KFC in the house. (Alleged)
f) There was KFC in the house. (Alleged, Confirmed)
g) Allison ate the KFC in the house. (Alleged)
h) Allison was not at the house. (Reported)
Rejected Hypotheses:
b) There was no KFC in the house, because it was only alleged and is mutually exclusive to h) that has a higher credibility of evidence.
c) Time was never in the house, for lack of confirmatory factors.
d) Brad was never in the house, for lack of confirmatory factors.
g) Allison ate the KFC in the house, because it was only alleged, and is mutually exclusive to h) which has a higher credible of evidence being “reported.”
Accepted Hypotheses:
f) There was KFC in the house, because although initially alleged, it was later confirmed by non-partisan contemporaneous documentation.
You get the idea. In the next round let’s say an Instagram of Tim shows up, posted on the date in time within the house, holding a bucket of KFC smiling and saying, “Sure tastes good!” which is confirmatory of a), f) and refuting of b), c), and g); some of which we have already rejected.
ROUND THREE
Boundary of All Active Hypotheses:
a) Tim ate the KFC in the house. (Alleged, Admitted)
e) Brad ate the KFC in the house. (Alleged)
f) There was KFC in the house. (Alleged, Confirmed, Admitted)
Rejected Hypotheses:
b) There was no KFC in the house, because it was only alleged and is mutually exclusive to h) that has a higher credibility of evidence.
c) Time was never in the house, for lack of confirmatory factors.
d) Brad was never in the house, for lack of confirmatory factors.
e) Brad ate the KFC in the house, because Tim admitted he ate the KFC via Instagram posting.
g) Allison ate the KFC in the house, because it was only alleged, and is mutually exclusive to h) which has a higher credible of evidence being “reported.”
Accepted Hypotheses:
a) Tim ate the KFC in the house, because he admitted it via Instagram.
f) There was KFC in the house, because although initially alleged, it was later confirmed by non-partisan contemporaneous documentation.
We can stop here, because our accepted hypotheses now lie within a boundary that contains no other mutually exclusive hypotheses.
Keep in mind, we haven’t proven “beyond a reasonable doubt” that Tim ate the KFC. The Instagram picture may have been staged, the UberEats receipt forged, Brad and Alison in some dark conspiracy to implicate Tim in a totally unreasonable and unfair allegation of KFC-consumption.
But here is why the standard of evidence I’ve selected is credible, clear & convincing, and substantial. Because we’re working in a complex system, and the trick of complex systems is doubts can be asserted in about any decision made. This gets worse in partisan situations where there is already a pre-existing bias to doubt something (either confirming or refuting) because of partisan alignment. But the Congressional member dealing with this cannot simply “not make a decision,” they will be asked eventually to make votes: to advance to an impeachment resolution, to vote on charges, and even to vote in a Senate trial. In these situations, either a vote for or a vote against is a decision…abstaining or voting present may be the only path of non-decision making and most constituents, regardless of side, are not going to be happy with an elected office holder punting on something like this.
None of us are in a position to make these calls, but if I were advising a Representative or a Senator, this is the methodology I might recommend. This wouldn’t remove any doubts, but it would provide an objective, reproducible method of assessing factors to arrive at a decision that could be defended later on and minimizes (without eliminating) the risk of making an incorrect decision based on a clear error.
Sources
Here are the media bias sites I’ll be using. These are pretty neat in that they’re no longer just static in nature but can be used interactively to seek out stories as well. The first is AdFontes
www.adfontesmedia.com/interactive-media-bias-chart/
And here’s MediaBiasFact Check:
www.mediabiasfactcheck.com/
A little more on the specific methods for mathematical heuristic factor analysis vs. component factor analysis, aka “Boundary & Factor Analysis.”
http://node101.psych.cornell.edu/Darlington/factor.htm
Original Facebook Post
www.facebook.com/tim.clancy.313/posts/10211041847562704
About Author
