How can I tell the combobox that it should always draw a pushbutton and not only for the current cell?

You may set

m_bInactiveDrawButton = TRUE;

This is a new attribute in the CGXComboBox and CGXComboBoxWnd class.

See the answer to the previous question  how to change this attribute for the combobox control.