When using SetSpecialMode(GX_MODELBOX_MS), is there a way to disable the ability to click on a row and extend the selection by moving the mouse over other rows?

Yes, you could override the CanChangeSelection method and return FALSE as shown in the following example:

BOOL CMyGrid::CanChangeSelection(CGXRange* pRange, BOOL bIsDragging, BOOL bKey)
{
   if (!bKey && pRange && pRange->GetHeight() > 1)
      return FALSE;
   return TRUE;
}