All posts
Liferay · 12 min read · Sep 29, 2026

Tuning Search for Large Content Sets

Tuning a search cluster for a large portal

Assuming the indexing pipeline and how queries are built are already familiar, this is about what happens when the content set gets large: what slows down, in what order, and which decisions you will wish you had made differently.

The uncomfortable part is that the highest-impact choices - how documents are mapped and how text is analysed - are made at the start, when there is no data to reveal whether they were right, and changing them later requires rebuilding the index.

What degrades, in order

OperationHow it scalesTypical first symptom
Full reindexLinearly, and it becomes a maintenance windowA reindex that no longer finishes overnight
Deep paginationBadly - cost grows with the offsetLater pages time out while page one is fast
Permission-filtered queriesWith the number of roles involvedSlow for administrators, fine for ordinary users
Aggregations and facetsWith cardinality, not document countOne facet dominates the query time
HighlightingWith document sizeSlowness proportional to result size, not count
Simple keyword lookupVery wellRarely the problem, despite being blamed
The thing people optimise first - the basic query - is usually the thing that was never slow.

Reindexing is the operation that hurts

A full reindex is the search equivalent of a table rebuild: it reads everything, writes everything, and competes with live traffic while doing so. On a large portal it stops being an administrative action and becomes a planned event.

  1. Reindex specific types rather than everything, because the reason is nearly always one type.
  2. Schedule full reindexes for low-traffic windows and expect degraded search while they run.
  3. Build to a new index and switch over on completion, rather than emptying the live one first.
  4. Keep incremental indexing healthy, since a backlog there is what creates the urge to reindex everything.
  5. Measure how long it takes at current volume, and watch that number grow before it becomes a crisis.

Building into a fresh index and switching is the practice that changes this most. It turns a period of broken search into a period of slightly stale search, which is a different conversation with the business.

Mapping decisions you cannot easily undo

How a field is analysed determines what can match it, and changing that on an existing index means rebuilding it. Three decisions are worth extra care because they are expensive to revisit.

  • Language analysis: content in a known language benefits substantially from language-aware stemming, and a generic analyser quietly returns worse results rather than obviously wrong ones.
  • Which fields are searchable text and which are exact-match values: identifiers, codes and statuses should not be analysed as prose, or they will match partially and surprisingly.
  • Which fields need sorting or aggregation, since those require a different treatment from free-text search on the same content.

Over-indexing is its own cost. Every field indexed in several ways multiplies index size and slows writes, and a large share of the fields teams index defensively are never queried. Indexing what is actually searched is both faster and smaller.

Permission filtering is the portal-specific problem

General search advice does not cover this, and in a portal it dominates. Every query must be constrained to what the current user may see, so the filter is a function of that user's roles and group memberships - and a user with many of them produces a substantially larger query than a user with few.

The counter-intuitive consequence is that the people who report search being slow are usually administrators and power users, whose permissions are broadest. Testing with an ordinary account will not reproduce it, which is why the reports often go unconfirmed for weeks.

The practical mitigations are to keep role counts sane, to avoid granting permissions individually where a group would do, and to be wary of content structures that require the filter to enumerate large numbers of containers.

Do not offer page five hundred

Deep paging is expensive for the same reason it is expensive in a database: reaching a distant offset means producing and discarding everything before it. The engine will refuse beyond a limit, and the refusal is correct. This is the familiar pagination problem with a lower tolerance than usual.

The product answer is better than the technical one. Nobody browses to the five-hundredth page of results; they refine instead. Offering good filters and a cursor-style continuation for the cases that genuinely need to walk a result set removes the problem rather than optimising it.

Measure the right numbers

SignalWhy average values mislead
Query latency percentilesThe average hides the slow tail, which is what users report
Slow query logNames the actual queries rather than the feeling
Indexing queue depthA growing queue means results are increasingly stale
Heap usage and collection pausesSearch nodes fail here first, usually before disk or processor
Free memory outside the heapFile caching drives read performance more than heap size does
Index size growth over timeThe best early warning for reindex duration
Percentiles and queue depth predict trouble; averages report it afterwards.

Sizing is mostly a memory question. Beyond a modest heap, additional memory is better left to the operating system for caching index files, and adding heap past that point tends to make collection pauses worse rather than search faster.

Know when search is the wrong tool

A recurring pattern is a reporting requirement implemented as a search query, because the data is already indexed and the aggregation capability is right there. It works at small volume and degrades badly, because the index is optimised for finding a few relevant documents rather than for summarising all of them.

Dashboards that count, group and total across the entire content set belong in a reporting store built for that shape of question. Keeping search focused on retrieval, and moving analysis elsewhere, resolves a category of performance problem that no amount of cluster tuning will fix.

Conclusion

Search degrades in a predictable order: reindexing first, then deep paging, permission-heavy queries and high-cardinality aggregations, while simple retrieval stays fast throughout. Make the mapping and analysis decisions deliberately at the start, since they are the expensive ones to revisit. Reindex into a new index and switch. Test performance with the broadest permission set, not the narrowest. Watch percentiles and queue depth rather than averages. And when the requirement is really reporting, stop asking the search engine to be a reporting database.

Contact

Let’s connect

Have a Liferay or Backend project you’d like to talk through? Fill in the form and I’ll get back to you.

Book Calendar