Mark Reynolds on Friday lost an appeal against judge's decision not to recuse herself over an 11-year professional relationship with Mr Rayney's barrister.
Mr Rayney was acquitted of murder following a high-profile trial after his estranged wife Corryn was found buried in a shallow grave in 2007.
He successfully sued Dr Reynolds over comments made in front of about 120 people at a conference in 2014 - about two years after the not-guilty verdict.
The WA Supreme Court defamation trial in 2020 heard Dr Reynolds told the forum: "I was the chief supervising officer on the case and there's no need for a cold case review. The offender was identified."
He later reiterated his comments at the same gathering, allegedly shouting: "I would stake my kids' lives on it."
Before Justice Jennifer Hill handed down her decision, Dr Reynolds learned she and Mr Rayney's lawyer Martin Bennett had worked together 15 years before the trial.
He applied for Justice Hill to recuse herself but she rejected it and upheld Mr Rayney's defamation claim, awarding him $438,000 in damages.
If leave to appeal had been granted and the appeal allowed, Dr Reynolds would have sought orders to set aside the judgement.
Dr Reynolds had claimed Justice Hill made mixed errors of fact and law in assessing his recusal application and there was a reasonable apprehension of bias arising from the relationship.
He also said various trial management decisions took on a different interpretation after he learned about the relationship and he would have responded differently had he known about it.
Alleged "concerning actions" by Mr Bennett, including disparaging behaviour, were also highlighted.
But the Appeal Court found Justice Hill had made the correct decision to dismiss Dr Reynold's recusal application and refused to leave to appeal.
Mr Rayney was previously awarded a $2.6 million payout for being defamed by police when he was named the "prime" and "only" suspect in his wife's murder.
In 2020, he was struck off as a lawyer after he was found to have secretly recorded his wife's conversations and gave false evidence about it in court.
The murder of the mother-of-two remains unsolved.
12°C