By default, pagination is done on server side when you set: e.g.
$opt["rowNum"] = 100;
$g->set_options($opt);

Performance of loading data in grid consist of 2 parts.

1) Client side: This is usually due to browser rendering capability of large html table, and JS engine that fill html.
This can be improved by enabling virtual scrolling of grid. It loads data only when it is required by scrolling.
As per docs, When enabled, the pager elements are disabled and we can use the vertical scrollbar to load data.

$opt["scroll"] = true;
$g->set_options($opt);

2) Server side: This depends on database optimization factors, such as indexes, usage of 'like' queries etc.
Best approach to optimize is to create indexes of your most searchable fields and avoid '%blah%' contains query
which never use indexes in mysql. After creating indexes, you can try SELECT queries in phpmyadmin and track loading time.
If phpgrid loading time is still slow, drop me a message with you sql and i'll check the internals.