Monday Madness - Who's Your Daddy?

Female ancestors who were never married --- the madness of it all (genealogically speaking of course)!

My husband's 3rd great-grandmother Minerva Jane Mullins never married and had six children, one of which was James Calvin Calaway "Cad" Mullins born March 1854 in Tennessee.
James C. Mullins is somewhat of a mystery to me. Friends and family members knew him as "Cad", "Cade", or "Caid", depending on who is spelling it. He is listed as Chad in his marriage certificate with Annis. He is listed in the 1900 & 1910 census as James C. He is listed as J. C. in the 1920 census. He witnessed his daughter, Eva's marriage as Calvin Mullens. In the 1860 Hancock Co, TN census he is listed as Calaway, and the 1870 Hawkins Co, TN census, he is listed as Caloway. He is listed as Caid on Eva's social security application.

I am looking at the possiblity that the Thomas E. Brown that James Calaway is living with in 1870 could be his father. I haven't been able to find any other relationship between them.

1870 Hawkins Co, TN Census, Dist 2, Lee Valley, p 18a.
"18A 43 43 Brown, Thomas E. 48 farmer
18A " James 21 working on farm
18A Lawson, Orlena 19 house keeper
18A Frazier, Sallie 45
18A Mullins, Caloway 16 working on farm

I ordered a copy of Thomas E. Brown's will hoping that would clarify something, but it didn't really.

WILL:
Thomas E. Brown
Know all men by these presence that I Thos. E. Brown, sr, of the County of Hawkins & State of Tennessee, viewing the uncertainty of life and the certainty of death do make this my last will and testament, at my death I wish all my just debts and funeral expenses paid out of my estate.
I further will that my beloved wife, Mary M. Brown, shall have the use of the homestid houses her life time, also to have one horse & one cow and beds and furniture, and my will is that my son, Samuel M. Brown, shall have the one hundred acres tract of land I bought of Abiga Martain.
I further wish to divide my mountain farm I bouth of C. M. Starnes between my two sons, Sam'l M. Brown & Thomas Jesse Brown, beginning in the road where the road that they travel from Uncle Mat Cope's intersex the mountain Road, thence with said road toward Clinch mountain to a pair of draw bars called the meadow drawbars, thence towards Clinch mountain with fence that divides the meadow and what is called Born field to the uper end of meadow, thence with pathway of Mat Martain's passing near his chimney to the branch, thence up said branch and with said hollow to top of said mountain Martin Gap.
My wish is that my son, Thomas Jesse Brown shall have alll the East side of mountain farm not sold to Tip Jones and Calaway Cope and I wish Sam'l M. Brown, my son, to have the South side of farm as I have divided, and I wish my son Samuel M. Brown to live with his mother and take care of her while she lives and If Samuel M. Brown, my son, should fail to take care of his mother and support her she is to have her support of home stid or dowry; and should I have at my death cash on hands after my debts and funeral expenses is paid, I wish it Equally divided between my two sons, Thomas Jesse Brown and Sam'l M. Brown and daghter Alice Pearson and my wife Mary M. Brown, This is my last will and testament, revoking all other wills made by me through life, in witness of the above I hereunto set my hand and seal, the Aug 8th 1899.
Thos. E. Brown (Seal)
Witnesses:
James M. Brown
Robt. K. Cope Probated Aug 6th 1900

Then there is my x-husband's 5th great-grandmother Mary Jane Martin who was married to Alexander McCullough abt 1854 Douglas County, Missouri. Mary Jane was pregnant with her third child, James when she found her husband, Alexander dead on a brush pile.

She had five more children after Alexander's death including Thomas Monroe McCullough. She never remarried.

I haven't been able to find any clues as to who their daddys were!