Kuenssberg: I asked Earl Spencer who we should believe - him or the King?

When I arrived at Althorp on Thursday to speak to Charles, Earl Spencer, only a handful of people were in evidence in the vast stately home, where Princess Diana had once tapdanced in the hall and rollerskated in the cavernous library.

It was only hours after Buckingham Palace had sent out its blistering statement in response to the explosive claims made in a new book by Earl Spencer. From the particularly English peace of this grand countryside setting, Diana's brother had provoked a storm.

That Palace statement, questioning Earl Spencer's account of the days after his sister's death, was a total contrast to its normal "never explain, never complain" mantra. But it meant that as we sat down, with oil paintings of Earl Spencer's aristocratic ancestors staring down at our TV lights, the interview was not just about his recollections of his sister. It had become about whether the earl, or the King himself, was telling the truth. As we asked Earl Spencer: which Charles are we meant to believe?

That's why it was so important to test Earl Spencer's claims. Was he really sure that the then-Prince Charles had sounded "giddy" after his former wife had died on that terrible night in 1997? Both men, after all, were in a moment of profound shock.

Did the earl, a historian and former journalist himself, have evidence of another of his memories, that Charles had "hissed" at him that the world would "forget her soon enough"? Why does he, the uncle of William and Harry, think the world needs to hear his version of events from all those years ago?

Earl Spencer was resolute; strong in his conviction that his version of events needed to be told as a matter of record. And he recounted his memories of his conversations with the King in meticulous detail, noting carefully in our discussion that the Palace had not explicitly denied his claims, instead choosing to question his judgement.

There is deep unhappiness and frustration in Buckingham Palace at what Earl Spencer has put into the public domain. Their strongly worded statement tells you everything you need to know about how upset they are. "While we do not comment on books as a matter of principle, His Majesty is mindful that the pain of fraternal grief can cloud reason, affect judgment and colour memory in ways others do not recognise, even many years after such a loss," it said.

Bear in mind that the two Charleses have had a difficult relationship for decades. Even before either of them was born, the Spencers and the Windsors, two aristocratic families, had years of splits and spats. In the past 48 hours, allies on either side have lined up to back the two different accounts, with acolytes of the King suggesting he would never have said such things. Others emerged to back Spencer.

There is no way of being completely sure what exactly happened. There hasn't been an explicit denial, but the Palace has objected in very strong terms to how Earl Spencer has presented the past. But speaking to Diana's brother for about half an hour, I felt he seemed entirely sure in his own mind. His book is carefully constructed to give his account, as well as pay tribute to his sister's life as the 30th anniversary of her death approaches.

After we had spoken, we walked down to the lake on the estate where, on a wooded island, Diana was laid to rest. It's a place where, Earl Spencer told me, she could be safe from intrusion, and is marked as a burial place with a huge stone urn.

We passed the little green rowing boat he uses to go to the island to visit her, and walked to what he calls The Temple. It's an 18th Century building that now bears her name and includes a quote from her and another from his famous eulogy to her. It's a line he now says was specifically intended to be a riposte to the other Charles' suggestion that she would quickly be forgotten: Diana "will never be extinguished from our minds".

Earl Spencer told me he was not being "vengeful" and that he was not trying to damage the monarchy - even saying there was more he knew about the King that he could relate but had chosen not to. But by speaking out so boldly, publishing his book, serialising it in the papers, and speaking to us, the earl was not just paying a moving tribute to his sister, but stirring up the long-running contest for control of Diana's memory.

In the closing moments of our interview, Earl Spencer revealed himself not to be a public figure trying to settle old scores or correct myths about his incredibly famous sister. When I asked how much he missed her, he welled up, and nearly three decades on, the tears still came. He said he missed her every day. "The thing with sibling loss? You never get over it," he said.

Charles Spencer was a young man when his sister died at 36 in the most awful circumstances. He joked about crying as a 62-year-old man at the end of a TV interview in which he'd discussed the monarchy and also made claims about the King that he was well aware would stir up controversy around the world. In the normally hidden world of Althorp, the grief of a brother was still plainly on display – what happened to Princess Diana is not all in the past.