Serbian tennis ace has sights set on Tokyo Games as he seeks his first Olympic gold.
At last November's Davis Cup Finals, the Serb said Olympic gold – the only major achievement he has yet to win in his career – was "one of the dreams".
Asked by Murray which result from his career he would change if he could, Djokovic said: "I think it would be related to Olympic Games.
"Maybe the match I played against you in London, or Rafa [Nadal] in Beijing 2008. It was very close [in 2008], it was 3-3 or 4-4 and 15-30, something like that, and I lost 6-4 in the third set.
"For sure, I was very fortunate to have the great success in my career and win all four Slams and all Masters series.
"I did win bronze in Beijing, but… if I had a chance to turn back time and change the outcomes, it would probably be Rio or London."
At the Davis Cup Finals, in response to a question from Olympic Channel, Djokovic said that Tokyo was high on his priorities for 2020.
"I'm going to try to be healthy, be fit, be prepared to play my best. Olympics are always bringing something extraordinary," he said.
"Every four years you get a chance to be part of the most historic sports events in the history of sport. You never get really a chance to, other than in the Olympics, to really sit next to all the elite athletes from their sports and dine with them and just exchange experiences and knowledge and everything."
The closest he has come was winning bronze in Beijing after losing to eventual champion Nadal in the semi-finals.
Four years later, Murray defeated the Serb on the courts of Wimbledon, again in the last four.
And in Rio 2016, a combination of a last-minute injury and a difficult draw saw Djokovic eliminated in the first round.Explaining his early exit in Brazil at the hands of Juan Martin del Potro, who would go on to lose to Murray in the final, Djokovic said: "I remember two days before the match I was practising, and I was feeling great.
"I'd lost third round at Wimbledon so I had enough time to get ready, I went and won Canada and I came to Rio full of confidence. I had the 15 best months of my career behind me before Rio.
"I remember I was practising singles for two hours on centre court, and then I went on with the Serbian guys to practise for another hour, hour and a half, doubles, and I started to feel my wrist at the end of the doubles sets that we played.
"I felt really sad that I wasn't at my best and I had a really tough draw, and I felt like maybe if I had a slightly better draw I could kind of work my way through the tournament."