Comments on news posts: one forum discussion per post

oc:pages command in the forum extension, tools/forum_sync.py and 'make forum-sync' (run by deploy) keep data/forum_pages.json up to date; post pages end with their discussion. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YDRjtmAiNpo5mK5N3d7QwZ
2026-10-11 22:22:20 +02:00
parent 418913d47e
commit 785c4967be
12 changed files with 243 additions and 31 deletions
@@ -6,8 +6,13 @@ PAGEFIND ?= npx -y pagefind@1
 # test server: the web share of barbel-lescar, mounted from the file manager (smb://barbel-lescar/web)
 DEPLOY_DIR ?= /run/user/$(shell id -u)/gvfs/smb-share:server=barbel-lescar,share=web/oldcomputers
 DEPLOY_URL ?= https://barbel.synology.me/oldcomputers/
+# forum on the server (comment discussions of the news posts, see forum-sync): FORUM_RUN runs a
+# shell line where the forum lives; empty = no forum, deploy then skips the sync
+FORUM_RUN ?= ssh -p 27 yannick@192.168.1.98
+FORUM_DIR ?= /volume2/web/oldcomputers-forum
+FORUM_PHP ?= /usr/local/bin/php84 -d memory_limit=512M

-.PHONY: content build check serve deploy post forum-up forum-import forum-down forum-bundle
+.PHONY: content build check serve deploy post forum-up forum-import forum-down forum-bundle forum-sync

 content:       ## regenerate content/ and static/media/ from the extraction repo
 	python3 $(OC)/scripts/05_hugo.py --target $(CURDIR) $(if $(wildcard data/forum.json),--forum)
@@ -27,7 +32,12 @@ post:          ## new post: make post NAME=my-title, then edit content/post/<tod
 	@test -n "$(NAME)" || { echo "usage: make post NAME=my-title"; exit 1; }
 	hugo new content post/$(shell date +%F)-$(NAME).md

-deploy: check  ## build, check, and copy $(BUILD)/ to DEPLOY_DIR (served at DEPLOY_URL)
+forum-sync:    ## create the forum discussion of each new news post (updates data/forum_pages.json)
+ifneq ($(strip $(FORUM_RUN)),)
+	python3 tools/forum_sync.py --run '$(FORUM_RUN)' --dir '$(FORUM_DIR)' --php '$(FORUM_PHP)' --site-url '$(DEPLOY_URL)'
+endif
+
+deploy: forum-sync check  ## build, check, and copy $(BUILD)/ to DEPLOY_DIR (served at DEPLOY_URL)
 	@test -d "$(dir $(DEPLOY_DIR))" || { echo "share not mounted: $(dir $(DEPLOY_DIR))"; exit 1; }
 	mkdir -p "$(DEPLOY_DIR)"
 	rsync -r --size-only --info=stats1 $(BUILD)/media/ "$(DEPLOY_DIR)/media/"
@@ -10,7 +10,7 @@ Static rebuild of old-computers.com with [Hugo](https://gohugo.io/) (0.158 or ne
 | `static/media/` | generated (copied images and documents) |
 | `data/slugs.json` | generated, keeps URLs stable between runs — do not delete |
 | `content/post/` | written by hand: news posts, see below |
-| `data/forum.json` | machine → forum thread map, written by the forum import (absent = no forum) |
+| `data/forum.json`, `data/forum_pages.json` | machine → forum thread and news post → forum thread maps, written by the forum import and by `make forum-sync` |
 | `forum/` | forum extension, local test stack, install notes |
 | `content/_index.md`, `layouts/`, `assets/`, `hugo.toml`, `tools/` | written by hand |
 | `build/`, `public/` | build output (`make build`, `hugo server`), not versioned |
@@ -32,6 +32,10 @@ banner, date, author and summary (`summary:` in the front matter, or the text be
 author's page, `listPosts` per page (20 by default). Posts are never touched by
 `make content`.

+With a forum, each published post gets a comment discussion, shown under the post. The
+discussion is created by `make forum-sync`, which `make deploy` runs first: it lists the posts,
+has the forum create the missing discussions and records them in `data/forum_pages.json`.
+
 ## Search

 Search runs in the visitor's browser on an index generated by [Pagefind](https://pagefind.app/)
@@ -0,0 +1,18 @@
+{
+ "post/2026-09-11-how-the-site-is-being-rebuilt.md": 14,
+ "post/2026-09-13-stupid-scans-are-back.md": 15,
+ "post/2026-09-15-when-commodore-introduced-the-pet.md": 16,
+ "post/2026-09-17-the-mother-of-all-demos.md": 17,
+ "post/2026-09-19-the-8086-the-chip-behind-the-pc.md": 18,
+ "post/2026-09-21-kaypro-loyal-to-its-customers.md": 19,
+ "post/2026-09-23-oric-disks-on-a-pc-and-back.md": 20,
+ "post/2026-09-25-a-hard-disk-for-your-8-bit-atari.md": 21,
+ "post/2026-09-27-calling-at-1200-baud.md": 22,
+ "post/2026-09-29-more-sound-for-the-c64-the-sound-expander.md": 23,
+ "post/2026-10-01-the-first-home-console-the-magnavox-odyssey.md": 24,
+ "post/2026-10-03-the-apple-ii-returns-to-the-museum.md": 25,
+ "post/2026-10-05-space-invaders-the-game-that-emptied-pockets.md": 26,
+ "post/2026-10-07-inside-the-c64-a-look-at-the-mainboard.md": 27,
+ "post/2026-10-09-the-commodore-64-pages-are-online.md": 28,
+ "post/2026-10-11-we-are-back.md": 29
+}
@@ -6,7 +6,7 @@ frame.

 | Path | Role |
 |---|---|
-| `extension/` | Flarum extension `oc/flarum-museum`: the `oc:import` command and the look of embedded threads |
+| `extension/` | Flarum extension `oc/flarum-museum`: the `oc:import` and `oc:pages` commands, original author names, the look of embedded threads |
 | `docker/`, `docker-compose.yml` | local test forum; `docker/bundle.sh` packages the server bundle |
 | `import/` | not versioned: `machines.json` (export from the extraction repo), `forum.json` (thread map), `forum.tar.gz` (server bundle) |

@@ -33,6 +33,16 @@ frame.
 - With `params.forum.url` set, the bottom of the machine's pages shows `<forum>/embed/<id>` (Flarum's Embed
  extension) in a frame that follows the height of the thread.

+## News posts
+
+`php flarum oc:pages` creates one discussion per news post under the tag News (listed with the
+other forum topics, unlike the machine threads), opened by the system account `News` with a
+link to the post. `make forum-sync` in the website repository drives it: `tools/forum_sync.py`
+sends the list of published posts to the forum host (`FORUM_RUN`, `FORUM_DIR`, `FORUM_PHP` in
+the Makefile) and stores the answer in `data/forum_pages.json`. `make deploy` runs it first, so
+a new post gets its discussion when it is published. A post that is renamed gets a new
+discussion; a retitled one keeps its discussion and its old title there.
+
 ## Local test

    make forum-up        # http://localhost:8081, admin / adminadmin (first start takes a minute)
@@ -2,10 +2,11 @@

 use Flarum\Extend;
 use Oc\Museum\Console\ImportCommand;
+use Oc\Museum\Console\PagesCommand;
 use Oc\Museum\DisplayName;

 return [
-    (new Extend\Console())->command(ImportCommand::class),
+    (new Extend\Console())->command(ImportCommand::class)->command(PagesCommand::class),

    // selected by oc:import (setting display_name_driver = museum); needs the Nicknames extension
    (new Extend\User())->displayNameDriver('museum', DisplayName::class),
@@ -0,0 +1,11 @@
+<?php
+
+use Flarum\Database\Migration;
+use Illuminate\Database\Schema\Blueprint;
+
+// One discussion per commentable page of the site other than machines (news posts);
+// page_key is the page's content file, e.g. "post/2026-10-11-we-are-back.md".
+return Migration::createTable('oc_page_threads', function (Blueprint $table) {
+    $table->string('page_key', 190)->primary();
+    $table->unsignedInteger('discussion_id')->unique();
+});
@@ -43,7 +43,7 @@ class ImportCommand extends AbstractCommand
        'Pongs' => ['#15803d', 'fas fa-table-tennis-paddle-ball'],
    ];

-    public function __construct(private ConnectionInterface $db, private SettingsRepositoryInterface $settings)
+    public function __construct(protected ConnectionInterface $db, protected SettingsRepositoryInterface $settings)
    {
        parent::__construct();
    }
@@ -75,7 +75,7 @@ class ImportCommand extends AbstractCommand
        $this->settings->set('display_name_driver', 'museum');

        $site = rtrim((string) $this->input->getOption('site-url'), '/');
-        $archive = $this->archiveUser();
+        $archive = $this->systemUser();
        $this->group = $this->group();
        $this->loadAuthors();
        $tags = $this->tags();
@@ -138,7 +138,7 @@ class ImportCommand extends AbstractCommand
        return $discussion->id;
    }

-    private function post(Discussion $discussion, User $user, Carbon $when, string $markdown): CommentPost
+    protected function post(Discussion $discussion, User $user, Carbon $when, string $markdown): CommentPost
    {
        $post = new CommentPost();
        $post->discussion_id = $discussion->id;
@@ -202,7 +202,7 @@ class ImportCommand extends AbstractCommand
    /** --replace: forget imported threads in which only archive accounts have posted. */
    private function removeUntouchedThreads(): void
    {
-        $archived = $this->group->users()->pluck('users.id')->push($this->archiveUser()->id)->all();
+        $archived = $this->group->users()->pluck('users.id')->push($this->systemUser()->id)->all();
        $removed = $kept = 0;
        foreach ($this->db->table('oc_threads')->pluck('discussion_id', 'oc_id') as $ocId => $discussionId) {
            $discussion = Discussion::find($discussionId);
@@ -232,14 +232,14 @@ class ImportCommand extends AbstractCommand
        return $group;
    }

-    private function archiveUser(): User
+    protected function systemUser(string $name = self::ARCHIVE_USER): User
    {
-        $user = User::query()->where('username', self::ARCHIVE_USER)->first();
+        $user = User::query()->where('username', $name)->first();
        if (! $user) {
            $user = new User();
-            $user->username = self::ARCHIVE_USER;
-            $user->email = 'archive@invalid.invalid';
-            $user->password = Str::random(40);  // nobody logs in as the archive
+            $user->username = $name;
+            $user->email = Str::lower($name).'@invalid.invalid';
+            $user->password = Str::random(40);  // nobody logs in as a system account
            $user->is_email_confirmed = true;
            $user->joined_at = Carbon::now();
            $user->save();
@@ -252,19 +252,24 @@ class ImportCommand extends AbstractCommand
    private function tags(): array
    {
        $tags = [];
-        $position = (int) Tag::query()->max('position');
        foreach (self::TAGS as $name => [$color, $icon]) {
-            $tag = Tag::query()->where('slug', Str::slug($name))->first();
-            if (! $tag) {
-                // hidden from "All discussions": one thread per machine would bury the forum topics
-                $tag = Tag::build($name, Str::slug($name), "Comments on the $name pages of the museum", $color, $icon, true);
-                $tag->is_primary = true;
-                $tag->position = ++$position;
-                $tag->save();
-            }
-            $tags[$name] = $tag;
+            // hidden from "All discussions": one thread per machine would bury the forum topics
+            $tags[$name] = $this->tag($name, "Comments on the $name pages of the museum", $color, $icon, true);
        }

        return $tags;
    }
+
+    protected function tag(string $name, string $description, string $color, string $icon, bool $hidden): Tag
+    {
+        $tag = Tag::query()->where('slug', Str::slug($name))->first();
+        if (! $tag) {
+            $tag = Tag::build($name, Str::slug($name), $description, $color, $icon, $hidden);
+            $tag->is_primary = true;
+            $tag->position = (int) Tag::query()->max('position') + 1;
+            $tag->save();
+        }
+
+        return $tag;
+    }
 }
@@ -0,0 +1,72 @@
+<?php
+
+namespace Oc\Museum\Console;
+
+use Carbon\Carbon;
+use Flarum\Discussion\Discussion;
+use Symfony\Component\Console\Input\InputArgument;
+use Symfony\Component\Console\Input\InputOption;
+
+/**
+ * Creates the discussion in which a page of the site is commented, for pages that are not
+ * museum machines: the news posts. Input is the list written by tools/forum_sync.py of the
+ * website repository. Pages that already have a discussion (table oc_page_threads) are skipped.
+ */
+class PagesCommand extends ImportCommand
+{
+    private const SYSTEM_USER = 'News';
+
+    protected function configure(): void
+    {
+        $this->setName('oc:pages')
+            ->setDescription('Create the comment discussions of the site\'s news posts')
+            ->addArgument('file', InputArgument::REQUIRED, 'pages.json written by tools/forum_sync.py')
+            ->addOption('site-url', null, InputOption::VALUE_REQUIRED, 'Address of the static site, e.g. https://example.org/oldcomputers', '')
+            ->addOption('map', null, InputOption::VALUE_REQUIRED, 'Where to write the page -> discussion map (forum_pages.json)');
+    }
+
+    protected function fire(): int
+    {
+        $pages = json_decode((string) file_get_contents($this->input->getArgument('file')), true);
+        if (! is_array($pages)) {
+            $this->error('Cannot read the pages file.');
+
+            return 1;
+        }
+        $site = rtrim((string) $this->input->getOption('site-url'), '/');
+        $user = $this->systemUser(self::SYSTEM_USER);
+        // unlike the machine threads, news discussions are listed with the other forum topics
+        $tag = $this->tag('News', 'Comments on the news posts of the site', '#b45309', 'fas fa-newspaper', false);
+        $done = $this->db->table('oc_page_threads')->pluck('discussion_id', 'page_key');
+        $created = 0;
+
+        foreach ($pages as $page) {
+            if (isset($done[$page['key']])) {
+                continue;
+            }
+            $this->db->transaction(function () use ($page, $site, $user, $tag, &$done) {
+                $when = Carbon::parse($page['date'] ?: 'now');
+                $discussion = Discussion::start($page['title'], $user);
+                $discussion->created_at = $when;
+                $discussion->save();
+                $first = $this->post($discussion, $user, $when,
+                    "Comments on the news post [{$page['title']}]({$site}{$page['path']}).");
+                $discussion->setFirstPost($first)->refreshCommentCount()->refreshLastPost()->refreshParticipantCount()->save();
+                $discussion->tags()->sync([$tag->id]);
+                $this->db->table('oc_page_threads')->insert(['page_key' => $page['key'], 'discussion_id' => $discussion->id]);
+                $done[$page['key']] = $discussion->id;
+            });
+            $created++;
+        }
+
+        $user->refreshCommentCount()->refreshDiscussionCount()->save();
+        $tag->discussion_count = $tag->discussions()->count();
+        $tag->refreshLastPostedDiscussion()->save();
+        if ($map = $this->input->getOption('map')) {
+            file_put_contents($map, json_encode($done, JSON_PRETTY_PRINT | JSON_UNESCAPED_SLASHES)."\n");
+        }
+        $this->info("$created discussions created; ".count($done).' pages mapped.');
+
+        return 0;
+    }
+}
@@ -5,13 +5,7 @@
 {{ if and $p.forum_thread site.Params.forum.url }}
 <section class="comments-section" id="comments" data-pagefind-ignore>
  <h2>Comments</h2>
-  {{ $thread := printf "%s/embed/%d" (strings.TrimSuffix "/" site.Params.forum.url) (int $p.forum_thread) }}
-  <p class="note">The first comments were left by visitors of the original site. Log in to the forum to add yours. <a href="{{ replace $thread "/embed/" "/d/" }}">Open this discussion in the forum</a>.</p>
-  <iframe class="forum-thread" id="forum-thread" src="{{ $thread }}" title="Comments on {{ .Title }}" loading="lazy"></iframe>
-  {{ with resources.Get "js/iframeResizer.min.js" | fingerprint }}
-  <script src="{{ .RelPermalink }}"></script>
-  <script>iFrameResize({ checkOrigin: false, heightCalculationMethod: "lowestElement" }, "#forum-thread");</script>
-  {{ end }}
+  {{ partial "forum-thread.html" (dict "id" $p.forum_thread "title" .Title "note" "The first comments were left by visitors of the original site. Log in to the forum to add yours.") }}
 </section>
 {{ else if $p.comments }}
 <section class="comments-section" id="comments" data-pagefind-ignore>
@@ -0,0 +1,9 @@
+{{/* A forum discussion shown in a frame sized to its content.
+     dict "id" <discussion number> "title" <page title> "note" <sentence shown above> */}}
+{{ $thread := printf "%s/embed/%d" (strings.TrimSuffix "/" site.Params.forum.url) (int .id) }}
+<p class="note">{{ .note }} <a href="{{ replace $thread "/embed/" "/d/" }}">Open this discussion in the forum</a>.</p>
+<iframe class="forum-thread" id="forum-thread" src="{{ $thread }}" title="Comments on {{ .title }}" loading="lazy"></iframe>
+{{ with resources.Get "js/iframeResizer.min.js" | fingerprint }}
+<script src="{{ .RelPermalink }}"></script>
+<script>iFrameResize({ checkOrigin: false, heightCalculationMethod: "lowestElement" }, "#forum-thread");</script>
+{{ end }}
@@ -5,6 +5,13 @@
  <p class="meta">{{ if not .Date.IsZero }}<time datetime="{{ .Date.Format "2006-01-02" }}">{{ .Date.Format "2 January 2006" }}</time>{{ end }}{{ with .GetTerms "author" }}{{ if not $.Date.IsZero }} · {{ end }}{{ partial "terms.html" (dict "page" $ "taxonomy" "author") }}{{ end }}</p>
  {{ with partial "banner.html" (dict "page" . "width" 1000) }}<img class="post-banner" src="{{ . }}" alt="">{{ end }}
  <div class="content">{{ .Content }}</div>
+  {{/* comments: the post's forum discussion, created by `make forum-sync` (data/forum_pages.json) */}}
+  {{ with and site.Params.forum.url .File (index (hugo.Data.forum_pages | default dict) .File.Path) }}
+  <section class="comments-section" id="comments" data-pagefind-ignore>
+    <h2>Comments</h2>
+    {{ partial "forum-thread.html" (dict "id" . "title" $.Title "note" "Log in to the forum to comment.") }}
+  </section>
+  {{ end }}
  <nav class="prev-next" data-pagefind-ignore>
    {{ with .PrevInSection }}<a href="{{ .RelPermalink }}">← {{ .Title }}</a>{{ end }}
    {{ with .NextInSection }}<a href="{{ .RelPermalink }}">{{ .Title }} →</a>{{ end }}
@@ -0,0 +1,71 @@
+#!/usr/bin/env python3
+"""Make sure every published news post has its comment discussion in the forum.
+
+Lists the posts with `hugo list all`, sends the list to the forum, where `php flarum oc:pages`
+creates the missing discussions, and stores the returned map (content file -> discussion number)
+in data/forum_pages.json, which the layouts read. Run before building (see `make forum-sync`).
+
+--run is the command that executes a shell line where the forum lives, e.g.
+  'ssh -p 27 user@host'  or  'docker compose -f forum/docker-compose.yml exec -T flarum sh -c'
+"""
+import argparse
+import csv
+import io
+import json
+import shlex
+import subprocess
+import sys
+from datetime import datetime, timezone
+from pathlib import Path
+from urllib.parse import urlparse
+
+ROOT = Path(__file__).resolve().parent.parent
+MAP = ROOT / "data" / "forum_pages.json"
+
+
+def posts():
+    """Published posts as the forum command expects them, oldest first."""
+    out = subprocess.run(["hugo", "list", "all"], cwd=ROOT, check=True, capture_output=True, text=True).stdout
+    now, pages = datetime.now(timezone.utc), []
+    for row in csv.DictReader(io.StringIO(out)):
+        if row["kind"] != "page" or row["section"] != "post" or row["draft"] == "true":
+            continue
+        date = datetime.fromisoformat(row["date"].replace("Z", "+00:00"))
+        published = datetime.fromisoformat(row["publishDate"].replace("Z", "+00:00"))
+        if published > now:
+            continue
+        pages.append({
+            "key": row["path"].removeprefix("content/"),
+            "title": row["title"],
+            "date": date.strftime("%Y-%m-%d %H:%M:%S") if date.year > 1 else "",
+            "path": urlparse(row["permalink"]).path,
+        })
+    return sorted(pages, key=lambda p: p["date"])
+
+
+def main():
+    ap = argparse.ArgumentParser(description=__doc__.split("\n")[0])
+    ap.add_argument("--run", required=True, help="command prefix that runs a shell line on the forum host")
+    ap.add_argument("--dir", required=True, help="forum folder on that host")
+    ap.add_argument("--php", default="php", help="PHP command line there")
+    ap.add_argument("--site-url", required=True, help="public address of the site, for the link in each discussion")
+    args = ap.parse_args()
+
+    pages = posts()
+    remote = (f"cd {shlex.quote(args.dir)} && cat > storage/oc-pages.json && "
+              f"{args.php} flarum oc:pages storage/oc-pages.json --site-url={shlex.quote(args.site_url.rstrip('/'))} "
+              f"--map=storage/oc-pages-map.json >&2 && cat storage/oc-pages-map.json; "
+              f"rm -f storage/oc-pages.json storage/oc-pages-map.json")
+    done = subprocess.run([*shlex.split(args.run), remote], input=json.dumps(pages), capture_output=True, text=True)
+    notes = [line for line in done.stderr.splitlines() if "discussions created" in line]
+    if done.returncode != 0 or not done.stdout.strip().startswith(("{", "[")):
+        sys.exit(f"forum sync failed:\n{done.stderr.strip()[-800:]}")
+    mapping = json.loads(done.stdout) or {}
+    text = json.dumps(dict(sorted(mapping.items())), indent=1) + "\n"
+    if not MAP.exists() or MAP.read_text() != text:
+        MAP.write_text(text)
+    print(f"{len(pages)} posts; {notes[-1].strip() if notes else 'done'}")
+
+
+if __name__ == "__main__":
+    main()