Skip to content

See the demo

Agoranomos comes with a demo: the history of 18 months of a fictional business with two clients, on four fictional platforms in the shapes of real APIs, with a fictional source of exchange rates, and a daily load that brings each new day. On the first start the demo data goes all the way through the stack, into the model of advertising statistics (The data model), so you can see what each part does before you connect a real ad platform.

What happens on the first start

  1. ./agoranomos up starts the stack, and the one-shot job demo-init starts once Airflow and Metabase are ready.
  2. demo-init unpauses the DAGs agoranomos_demo_history and agoranomos_demo_transform in Airflow and runs the history: its five tasks, one after another, load 18 months up to yesterday, in the time zone of each account, into the schemas raw_demo_search, raw_demo_social, raw_demo_video and raw_demo_web of the warehouse, in the shapes of the Google Ads API v25, the Meta Marketing API v26.0, the TikTok API for Business v1.3 and the GA4 Data API v1beta, and the exchange rates into raw_demo_fx.
  3. Each load updates the asset of the main table of its platform, and the DAG agoranomos_demo_transform runs when all five have loaded. Its first task loads the settings of the model: the clients of the demo, and the files of the deployment in extensions/model/ (Configuration). Then dbt, through Cosmos, builds the model of advertising statistics (ADR-0025): the seeds, the staging views of each platform, the steps that the platforms share, and the marts, such as marts.ad_stats_campaign_daily. Each model and each seed is one task, which builds it and then runs its tests, and a failed test stops the models that read it.
  4. demo-init unpauses the DAG agoranomos_demo_daily, which brings the new days every day (Keep the demo up to date).
  5. demo-init syncs the warehouse in Metabase, applies the metadata of the marts, and creates the collection Agoranomos, with the collection Demo in it, its questions and two dashboards, Advertising overview (demo) and Web analytics (demo), whose questions read marts.ad_stats_campaign_daily, marts.ad_conversions_campaign_daily and marts.web_sessions_daily. Every group but Administrators gets at most View on them.

up waits for demo-init, which gives up after 10 minutes. When it fails, up still waits for the rest of the stack, prints the addresses of Airflow and Metabase, says that the stack is up and the demo is not ready, and exits with 1. ./agoranomos logs demo-init shows one line that names the step that failed or ran out of time, and for a step of the pipeline the DAG to look at in Airflow. Fix the cause and run ./agoranomos up again: the next start does what is not done yet.

Look at it

  • Metabase, at the address that up prints: sign in with AGORANOMOS_ADMIN_EMAIL and METABASE_ADMIN_PASSWORD from .env (./agoranomos credentials shows the address, the email and the setting that holds the password), and open the collection Agoranomos / Demo. Its two dashboards show money in the reporting currency, AGORANOMOS_REPORTING_CURRENCY of .env (USD by default):

    • Advertising overview (demo): the spend, the conversions of the primary conversion actions, the cost per conversion and the return on ad spend; the spend of each day on each platform, and of each client; the impressions, clicks, click-through rate, spend, conversions, cost per conversion and return on ad spend of each platform; the 20 campaigns with the most spend; and the conversions and the conversion value of each conversion action, with its category and whether it is primary.
    • Web analytics (demo): the sessions, the engagement rate, the key events and the revenue; the sessions of each day in each default channel group, and the results of each channel group; the 20 pairs of a source and a medium with the most sessions; and the sessions of the campaigns that web analytics gives an ad platform.

    The filters of a dashboard, Period, Client and, on the advertising, Platform, set every card of it. The period starts as the past 30 days, which end yesterday, the last day that the daily load has brought (Keep the demo up to date). A chart with one series uses the first chart color of the project. - Airflow, at the address that up prints: sign in as admin with AIRFLOW_ADMIN_PASSWORD from .env, and open the DAGs agoranomos_demo_history, whose five tasks load one platform each, agoranomos_demo_transform, agoranomos_demo_daily, their runs and the assets between them.

What later starts do

demo-init runs on every start, and each step looks at what is there first.

  • Once a run of agoranomos_demo_history has succeeded and a run of agoranomos_demo_transform has succeeded after it, the pipeline does not run again, and demo-init leaves both DAGs as they are: if you pause them, they stay paused. demo-init unpauses agoranomos_demo_daily on the first start that finds it without a run, after the pipeline: the first start of the demo, and the first start after an update that brings the DAG, whose first run then loads the days since the history. Once it has a run, demo-init leaves it as it is too. A history without a good transform after it, as after a failed critical test, runs again on the next start, which replaces the same days. A start that finds the transform of a good history still running, as after a start that ran out of time, waits for it, and runs the history again only if it fails. A stack of v0.1, whose transform ran before the history existed, runs both on its first start of this release, which also removes the demo of v0.1 from the warehouse (Upgrading from v0.1).
  • The demo is view-only for everyone but admins: on every start, metabase-init, and demo-init after it, give every group but Administrators at most View on Agoranomos and on every collection in it, Demo included, and demo-init does the same for each collection that it makes, right after it makes it (ADR-0018). So no account but an admin can put anything into them; should anything reach a new collection before that, demo-init puts the collection in the trash with it and stops with one line, and the next start makes the collection again.
  • Each start of demo-init puts back what differs from the code: the names and descriptions of the collections, the questions and the dashboards, where they are, the type and the whole query of each question, with the template tags of its filters, all its settings of display, such as the titles and the formats of the columns and the color of the charts, and the metadata of the columns of its answer, whose display names and descriptions people and AI clients read, and each card of each dashboard, whole: its question, its place, its settings, such as a click behavior, and its filter mappings, with the filters, their defaults and the tabs of the dashboard. A setting that someone added is put back too. It brings an item back from the trash, creates again one that was deleted for good, and logs each change, with the value that it replaced for a name or a description, which ./agoranomos logs demo-init shows; Metabase keeps the earlier version in the history of revisions of the item. It finds them by the entity IDs that Metabase gave them, not by name: a renamed question is still the question of the demo and gets its name back, and a collection of the same name that someone else made is left alone, as is anything else that an admin puts into Agoranomos or Demo. An admin's edit holds until the next start.
  • To keep a variant of your own, duplicate a dashboard, with copies of its questions, into a collection of your own, and change the copy. The questions are SQL, so changing the query of a copy needs the permission of native queries.
  • The demo collection of v0.1, Agoranomos demo at the root, moves into Agoranomos as Demo at the first start of v0.2.0, once. On a Metabase of v0.1 every account may make a collection at the root, so the name alone proves nothing: the demo of v0.1 is the collection of that name that holds the dashboard Ad spend (demo) that the account of demo-init made, the lowest by its ID if there are several. It keeps that dashboard, which becomes Advertising overview (demo), so the links to it keep working, and the questions of v0.1 that the same account made, which are put right as the code says and become three questions of that dashboard. Everything else in it, such as a question that you added or renamed, a sub-collection, a timeline with its events, or anything that another account made, goes to the root first, also what is in the trash from it, which stays in the trash, with one line for each in ./agoranomos logs demo-init by its kind and its ID, so that only the content of the package enters Agoranomos. Before it moves, it is locked at the root, so that no group but Administrators may curate it any more, and it must hold nothing else then: should anything have reached it before the lock, demo-init stops with one line that names the item by its kind and its ID, the collection stays at the root, view-only, and the next start takes that item out and moves the collection. A collection of that name that is not the demo stays where it is, and one line says so; so does the collection of v0.1 when it holds an item of a kind that demo-init does not take out, such as a document, also one in the trash, or a pulse. In both cases the demo is made anew. Copies of the questions in other collections are not touched. To stop all of this, turn the demo off.

Keep the demo up to date

The DAG agoranomos_demo_daily runs every day at 09:17 UTC, when the day before has ended in the time zone of every account of the demo. Its first task, due, reads what the loads have recorded and skips the run when every account has been loaded on its day already, as a second load on the same day would bring the same numbers. Otherwise its five tasks load each platform, one after another: for each account, the days from the day after the last that a load took, or from the first day of the reload window of the platform when that is earlier, to yesterday in the time zone of the account. The reload windows are the days whose numbers can still change (ADR-0025): 90 days on demo_search, 28 on demo_social and demo_video, 14 on demo_web and 7 on demo_fx. Each load replaces those days of each account, so no row is duplicated, and the transform follows when all five platforms have loaded.

A stack that was down catches up when it starts again: the daily load makes a run for the last time of its schedule that passed, which brings every day since the last load. The loads of the history and of the daily load run one at a time, in the pool demo_load of Airflow, which airflow-init sets on every start. To stop the daily load, pause agoranomos_demo_daily in Airflow; demo-init leaves it paused. To load the whole history again, trigger agoranomos_demo_history.

Turn the demo off

Set COMPOSE_PROFILES=core in .env before the first start, and up makes no demo: the demo DAGs stay paused in Airflow, the daily load too. After the first start the same setting only stops demo-init, whose container from its last run stays in ./agoranomos status. To remove that container, run ./agoranomos down once with the demo profile on, with COMPOSE_PROFILES=core,demo in .env: for example just before you set COMPOSE_PROFILES=core. The demo data stays in the warehouse and the collection Agoranomos / Demo stays in Metabase, view-only, as metabase-init keeps it on every start; an admin can move it to the trash there. If you turn the demo on again, the next start brings it back from the trash.

./agoranomos init never changes a setting that is in .env already, so a .env from before the demo keeps COMPOSE_PROFILES=core. To turn the demo on there, set COMPOSE_PROFILES=core,demo.