Select the text and click the small triangle next to the Highlight and Underline in the floating bar to select the color.

You can also right-click on the annotation, click the Property Setting option and set it in the properties panel that pops up on the right.