When building a Custom Report, the Group By option is available based on the field type; not all custom field types support grouping.
Multi-select dropdown fields can't be used for grouping because single records can have multiple values selected, making it impossible to assign it to a single group.
Text Box fields support limited grouping.
Dropdown, Radio, Checkbox and Date fields support grouping.
If you need to group report data by a field that is currently a Multi-select dropdown:
Review whether the field can be reconfigured as a single Dropdown (if one value is selected).
Use an alternative field of a supported type for grouping purposes.
Use filters instead of grouping to narrow down results by that field's values.
See Working with Custom Reports and Dashboards for more information.