I found the answer at https://meta.stackexchange.com/a/281010
You can retract flags by clicking flag again and click retract flag at the bottom. I noticed though that it is not available in the Android app so I had to do it through my mobile browser which was awkward as it wanted to switch to the app, but heyho it is retracted now.
You need to paste the whole question link into the browser address bar to go to the question in the browser. You can get that in the app by clicking 'share' and copy the address to clipboard.