Building a Site Initializer Generator: Turning a Live Site into Deployable Code
A site initializer is a Liferay site expressed as code: a folder of JSON, XML, FreeMarker templates and documents that Liferay reads to create a site from scratch. Once a site exists in that form, it can be committed, reviewed, built and deployed like anything else. New environments, demo instances, disaster recovery and onboarding a new developer all become a deploy instead of an afternoon of clicking.
The catch is that almost no site starts life as an initializer. It is built in the UI first - structures, templates, documents, web content, pages, menus - and only later does someone ask for it to be reproducible. At that point the options are to rebuild it by hand as initializer files, which takes days and is easy to get subtly wrong, or to move it with an export/import archive, which carries content between instances but produces nothing you can read, diff or review.
We are building a third option: a tool that connects to a running site, reads it through Liferay's headless REST APIs, and generates site initializer source code. This post covers how we are approaching it. It is not finished, and one of the things worth saying up front is that it is not meant to produce a project you deploy blind. It is meant to do the slow, mechanical eighty percent and hand you an honest list of the rest.
What the output has to look like
Liferay's site initializer reads a fixed folder layout and imports each resource type in a fixed order. A generated project needs to match that layout exactly, for the Liferay version it will run on.
site-initializer/
documents/group/... # files + .metadata.json
ddm-structures/*.xml # web content structures
ddm-templates/<name>/ # ddm-template.json + .ftl
journal-articles/... # article .json + .xml
layout-page-templates/... # display page templates
layouts/01_home/ # page.json + page-definition.json
site-navigation-menus.json
object-definitions/*.jsonThat folder is then packaged one of two ways: as an OSGi module that declares the site initializer capability, or - on 7.4 GA132 and newer - as a siteInitializer client extension that Liferay applies on deploy. The generator supports both and picks the options from the target version.
The real challenge: everything points at everything else by ID
Copying a site's content is the easy part. Making the copy portable is the hard part, because a Liferay site is a dense web of references and nearly all of them are database IDs that will be different on the next instance.
{
"siteId": 20121,
"contentStructureId": 45210,
"image": {
"url": "/documents/20121/45811/hero.png/6c1f0e2a-...?t=1718000000000"
},
"link": { "plid": 37, "layoutId": 4 }
}Group IDs, company IDs, structure IDs, template IDs, file entry IDs, plids, asset list IDs, object definition IDs - they appear in article content, fragment configuration, page definitions, navigation menus, CSS and inline HTML. It is the same problem as hardcoded IDs in SQL, at the scale of a whole site. A generator that copies them through produces a project that builds, deploys, and then renders broken images and dead links.
Site initializers solve this with placeholder tokens. Instead of an ID, the file contains a token such as [$GROUP_ID$] or [$DDM_STRUCTURE_ID:ARTICLE$], and Liferay substitutes the real value at import time. The generator's central job is therefore reference rewriting: find every environment-specific value and replace it with the right token, or say clearly that it could not.
Challenge: a document URL has many shapes
Documents were the first place the rewriting got harder than expected. The same file can be referenced in half a dozen forms depending on who inserted it and where, and the rewriter has to recognise all of them.
- Relative /documents/{groupId}/{folderId}/{title}/{uuid} URLs, and absolute ones that include the source host.
- Query strings: ?version= and &t= cache busters to drop, download=true and imagePreview=1 to keep.
- JSON-escaped forms (\/documents\/...) inside serialized content, and %2F-encoded forms inside attributes.
- CSS url(...) references in fragment and theme styles.
- Friendly document URLs and the older /c/document_library/get_file form.
- Adaptive Media URLs, which can be mapped back to the original file but lose the specific image variant.
Every variant resolves to the same thing: a token keyed by the document's path inside the initializer's documents folder, so Liferay can look the file up after it has imported it.
Challenge: placeholders are not one universal language
The obvious approach is a table of ID types and tokens. It does not survive contact with the source code, and this was the most important thing we learned early on. We stopped trusting documentation and examples for token syntax and read the site initializer implementation for each supported version instead.
- Not every file gets substitution. Liferay runs two replacement passes - context tokens like [$GROUP_ID$] and map tokens like [$DOCUMENT_URL:...$] - and each resource type gets neither, one or both. A token written into a file that gets no pass stays in the output as literal text.
- Map tokens only resolve if the step that registers them has already run. Pages are imported after web content, so a link-to-page field inside an article cannot use a layout token: it does not exist yet when the article is imported.
- Keys are not always what they look like. Template tokens are keyed by template name rather than template key, the layout token is the layoutId rather than the plid, and a structure's key is derived from its upper-cased name when it is read from XML.
- Tokens differ between versions. Some exist only on GA132 and newer, and the client extension packaging format does not exist on older 7.4 updates at all.
The rule we settled on: a token is only allowed in a version profile once it has been confirmed against the Liferay source for that version, with a reference to where. The generator never invents a token, and it never emits one into a file where Liferay will not substitute it.
Challenge: the headless APIs do not cover everything
We decided early that the tool reads the source site only through public headless APIs - no internal Java services, no database access. That keeps it safe to point at an instance and independent of internals that change between updates. It also means its reach is exactly the reach of those APIs.
On 7.4, that reach has gaps. Fragments, style books and collections have no headless endpoint that returns what an initializer needs. Master pages and page templates are available on newer updates, but in a different model from the page definitions an initializer expects. These are not things the generator can paper over, so it reports them instead.
What we built: the capabilities
The tool is a React UI and a Spring Boot service. Both run standalone during development and both package as Liferay client extensions - a custom element widget and a microservice - so the builder itself can live inside the portal it reads. Each export runs as a job through a fixed pipeline.
Connect OAuth2 client credentials, held only in service config
Probe read each API's openapi.json into a capability map
Plan topological sort of exporters; fail fast on cycles
Collect page through the APIs, register every entity
Serialize write the folder layout for the target profile
Rewrite IDs and URLs -> verified tokens, or report
Package OSGi module or client extension + export reportCapability probing instead of version checks
Rather than asking "which version is this?" and branching, the service reads the OpenAPI specification of each headless API and builds a map of the endpoints and fields that actually exist. Each exporter enables itself from that map. An instance that lacks an endpoint shows that entity type as unavailable instead of failing halfway through an export. The version still matters for the output, so the service suggests a target profile from what it finds and the user confirms it.
A reference registry
As each exporter collects entities, it registers them: entity type plus source ID maps to a stable key and an output path. Document URLs map to their path in the initializer. Entities that already carry an external reference code keep it; those that do not get a generated one. By the time rewriting starts, the registry can answer "what is 45210 on the other side?" for everything that was exported.
A rewrite engine that knows where tokens are allowed
Rewriting runs in layers: JSON-aware ID replacement first, so a groupId field is treated as a group ID rather than any number that happens to match; then document URLs in all their variants; then absolute site URLs; and finally a scan for anything that still looks like a leftover ID. Every rewrite is checked against the profile's rules for that file type.
{
"siteId": "[$GROUP_ID$]",
"contentStructureId": "[$DDM_STRUCTURE_ID:ARTICLE$]",
"image": {
"url": "[$DOCUMENT_URL:/site-initializer/documents/group/images/hero.png$]"
},
"link": { "plid": 37, "layoutId": 4 }
}Dependency ordering
Exporters declare what they depend on: documents before structures, structures before templates, templates before articles, articles before pages, pages before navigation. The job sorts them, adds missing dependencies with a warning when a user selects only part of a site, and fails fast with a clear message if it ever finds a cycle.
Version profiles and two output formats
Everything version-specific - which tokens exist, which files get which replacement pass, the folder layout, the packaging template - lives in a profile file rather than in code. Supporting a new update means writing and verifying a profile, not editing the rewriters.
An export report and a next-steps checklist
Every zip contains a human-readable report and a JSON version of it: what was exported, what was skipped and why, every unresolved reference with its file, JSON path, original value and a suggested action, plus warnings such as key collisions or very large files. The UI turns the report into an ordered, tickable checklist - prepare the target, review the project, deploy, finish by hand, verify.
A small round-trip script closes the loop: after deploying the generated project to a clean instance, it checks that everything the report claims was exported actually exists there. It is the kind of check that belongs in a pipeline eventually rather than on a developer's machine.
Credentials stay in the service
The service authenticates with OAuth2 client credentials using read-only scopes. The client ID and secret come only from environment variables or a git-ignored local config file. The UI never sees, sends or stores a credential, and the service listens only on localhost with CORS limited to the portal's origin.
A strong start, not a finished project
This is the part worth being explicit about. The generator gets you a correctly structured, buildable project with the bulk of the content and the bulk of the references already handled. It does not get you a project you can deploy without reading. Some things still have to be changed by hand in the generated source, and the report lists every one of them.
| Still manual | Why | What to do |
|---|---|---|
| Fragments, style books, collections | No headless API exposes them on 7.4 | Deploy them to the target first, or add them to the initializer by hand; the report lists every key pages depend on |
| Master pages and page templates | Available only in a different model on newer updates | Recreate them, or add the definitions by hand |
| Link-to-page fields in web content | Layout tokens are registered after articles are imported | Re-link the fields after the first import |
| plids, asset list, category, role and segment IDs in page definitions | No token can safely replace them in those files | Fix each one using the JSON path in the report |
| Hardcoded IDs inside FreeMarker templates | Template scripts get no token substitution | Replace them with lookups by key or name |
| Widget pages | Exported as page shells only | Reconfigure the widgets after import |
| Object relationships, actions, layouts, views, validations | Not exported yet | Add them to the definition JSON or configure them after import |
| Categories, extra title translations, the default display page flag | Not exported, or exported in the default language only | Re-apply after import |
| Users, roles, permissions, workflows | Out of scope by design | Handle through your normal provisioning |
Two habits make this work in practice. First, read the report before the code: it tells you exactly where to look, which turns "is this complete?" from a guess into a checklist. Second, treat the generated project like a pull request from a new teammate - review it before it is committed. It was generated from real content, and a person should decide what belongs in a repository.
The guiding rule behind the whole design is that the tool never silently leaves an ID in the output. A reference it cannot rewrite safely is reported, not guessed. A generator that is right ninety percent of the time and quiet about the rest is worse than one that is right eighty percent of the time and tells you precisely where the other twenty is.
Where it stands
The first three phases are in place: the foundation and first exporters, the rewrite engine, and pages with site structure. Generated projects build in both output formats, and the pipeline has been run end to end against a mock of the headless APIs. It has not yet been round-tripped against a real instance, and automated tests for the rewriters are still to come. Those two are the next things that will prove or break the design.
- A small companion OSGi module that exposes only what the headless APIs are missing - fragments, style books, master pages, page templates, collections - so the largest manual items can move into the generated output.
- More version profiles, each verified against source.
- Include and exclude filters for folders, file types and sizes.
- A command-line entry point so exports can run in CI.
What we have learned so far
- Exporting data is easy; exporting references is the project. Most of the design effort went into the registry and the rewriter, not the exporters.
- Read the source for the contract. Token syntax, which files get substituted, and the import order all live in the implementation, not the documentation.
- Probe for capabilities rather than branching on version numbers. It degrades gracefully where version checks break.
- Report rather than guess. An explicit unresolved reference costs five minutes; a silently wrong one costs a debugging session on the target instance.
- Aim for a strong starting point and say clearly what is left. That is a more useful promise than full automation, and a more honest one.
Conclusion
Turning a live Liferay site into site initializer code is mostly a reference problem: every ID and URL that ties content to one database has to become a token the next instance can resolve. Reading only through headless APIs, probing capabilities, registering every entity, rewriting against tokens verified in the source, and reporting everything that cannot be rewritten turns days of hand-work into a generated project and a checklist. It will not remove the manual work entirely - fragments, master pages, page links and template scripts still need a person - but it moves that work from rebuilding a site to reviewing one.