Can I have "What's this?" type context-sensitive help on a per cell basis?

You can register an user attribute in the grid, let's say IDS_UA_HELPID and assign this help id to cells, rows, or columns, e.g.

SetStyleRange(CGXRange().SetCols(1), 
CGXStyle().SetUserAttribute(IDS_UA_HELP, szhelpId));

To implement the "What's this" functionality, you have to process the WM_HELPHITTEST function which is described in MFC Technote 28.

To do this,

a) include afxpriv.h

b) add declaration

   afx_msg LRESULT OnHelpHitTest(WPARAM wParam, LPARAM lParam);

c) add to message map

   ON_MESSAGE(WM_HELPHITTEST, OnHelpHitTest)

d) and the implementation of this method

LRESULT COGLView::OnHelpHitTest(WPARAM, LPARAM lParam)
{
   CPoint ptTest(LOWORD(lParam), HIWORD(lParam));
   ASSERT_VALID(this);
   ROWCOL nRow = CalcClientRowFromPt(ptTest);
   ROWCOL nCol = CalcClientColFromPt(ptTest);
   LRESULT nID = 0;
   if (nRow != GX_INVALID && nCol != GX_INVALID)
   {
      TRACE("OnHelpHitTest: nRow %lu, nCol %lu\n", GetRow(nRow), GetCol(nCol));
      CGXStyle style;
      ComposeStyleRowCol(GetRow(nRow), GetCol(nCol), &style);
      
      nID = atol(style.GetUserAttribute(IDS_UA_HELP));
   }
   return nID;
}