Updates

Bunny: decide Volume vs Standard tier (more PoPs, faster first byte far from Volume PoPs) once we've run the numbers on cost and latency; also why cache HIT is low (Statistics: status codes, top uncached URLs)

Rewrite node deploy to wait for image after all the init steps to speed up runs

Set up cloud, watch memory agent on instance and in code

We get two emails one after the other and we should only get one

• User in overdue disabled grace period
• User account has been closed after for late payments

Should we do a hybrid static build into s3 for blog HTML JS and CSS only?

Data volume follow-ups (shrunk to gp3 on 9 Oct with config/host/data-volume/resize.sh; see the data-volume skill):

• From 16 Oct: delete the 8 snapshots of the old data volume vol-0a2e04d301e025e60 (deleted 9 Oct): the pre-resize snapshot snap-0e4fdce70e58e0da6 (the rollback until then) and the 7 daily DLM ones, which the policy stopped managing when the volume lost its Name tag
• Migrate static content to S3, then shrink the volume again with resize.sh
• Have resize.sh shrink redeploy the proxy itself (its data/static bind does not follow a swap)
• Can we test my assumption that the ephemeral instance store hard disk is faster than EBS discs?

disk alerts emails add

create skills which delegate cheap subagents to scan logs for issues:

• proxy logs (focussed on error)
• node container logs

adjust disk resize script to autosuggest values on run without args

• i.e. propose shrink if free space found via ssh is X or grow if Y

harden app against full disk

Our AWS bill is high because we have run out of reserved instance. Decide on the plan.

• switch over static files to s3
• can we have multiple smaller webservers with load balancing?
• Should we move to a single, AWS instance and make redis a container as well? This may be makes deployment harder perhaps we should consider moving to three AWS instances one for proxy one for node and one for redis we could still consider using a container on the redis instance?
• Safely and very carefully switch AWS account root user email to @blot.im

Redis host follow-ups (from #2041, new host since 8 Oct):

• From 15 Oct 2026: run config/redis/perf/fetch.sh redis blot and compare.js (a full day on the new host against the 8 Oct baseline), then uninstall the perf collectors: config/redis/perf/install.sh --uninstall redis blot
• Lock heartbeat on its own RESP2 connection, no overlapping EXTENDs, per-call timeout; give the shared client a command timeout or TCP user timeout
• Decide which app/sync/lock.js diagnostics to retire
• Remaining dropped write errors: write Redis before deleting files (updateCdnManifest, template/drop); await set() in resetToDrive; Dropbox disconnect.js and _assign.js dropFromMenu check the wrong error; write order in Blog.create; the list-ghosts restore path; error paths for background resave and rebuild; a callback for the messenger's setStatus; make Entry.set atomic
• Proxy: auto-ssl's lock and hook calls can reuse a timed-out Redis connection
• Deploy log archive (scripts/deploy/index.js): docker logs -t, outside /tmp, more than 3 files, size-capped; let config/redis/perf/fetch.sh read it
• Upgrade Redis (7 or Valkey), revisit AOF
• Offsite Redis backups (e.g. Backblaze B2) alongside S3, so a lost AWS account or region doesn't take the backups with it: upload from config/redis/bin/backup.sh, and let launch.sh/restore.sh restore from B2 too (#2041)
• Consider dropping Redis client-side caching (CLIENT TRACKING) in favour of the app's in-process LRU caches: on prod 10 clients have tracking on and the server's tracking table sits at its 1M-key cap (tracking-table-max-keys), so Redis is constantly evicting entries and sending invalidations

odt converter extracts every document's media into one shared _assets/media (image1.png etc. overwrite each other, served immutable): give it a per-document hash dir like docx

Proxy container follow-ups (cut over 8 Oct 2026, #1941)

• Check webhooks. SSE through the container, and that a known data/static file on cdn. has Cache-Control 1y and no Blot-Upstream
• After the next 03:46 logrotate: access.log/error.log still written after copytruncate, fail2ban counts still moving
• After the next wildcard renewal (/home/ec2-user/renew-wildcard.log): container reloaded, served cert fingerprint matches /etc/ssl/private/letsencrypt-domain.pem
• Custom-domain certs renew inside the container: none within ~20 days of expiry apart from domains that no longer point at Blot; identify-expiring-certs cron restarts the container cleanly
• Reboot the host once in a maintenance window: mount-instance-store mounts without reformatting, docker starts after it, the proxy comes back with cache/log mounts, :80/:443 answer
• HTTP/3 is not advertised (no Alt-Svc on either stack); decide whether to open UDP 443 and make fail2ban bans cover UDP too
• http://www.blot.im/ returns 404: confirm intended
• app/blog/static has no copy on the host, so those cdn. files fall through to Node: export them from the Node image on deploy, or accept
• cdn. responses carry Cache-Control twice (expires 1y + add_header), as on bare metal
• Purges during a blue/green overlap reach only one container (shared SO_REUSEPORT on :8077)
• Use the proxy container in local development instead of the nginx:alpine in scripts/development/proxy, so local.blot runs the production proxy config
• Does proxy container deploy check for enough free memory and disk on host before proceeding?
• Does proxy container deploy work in paralell with node container deploy with nothing breaking?
• Get proxy horizontally scalable and multi-region
• Can we add modern compression to proxy?
• Can the proxy remember bad urls and avoid repeatedly sending them to upstream?
• container equivalent for fail2ban?
• Tune proxy next upstream status code conditions to preserve zero-downtime-deployments but prevent expensive requests (timeout?)
• Add support for IPv6 and tell Raul
• Verify that nginx config serves as many CDN static files as possible
• Investigate whether openrestycache can be modified to pre-compress gzip responses
  • can we add support for brotli?
  • can we use https://nginx.org/en/docs/http/ngx_http_gzip_static_module.html#gzip_static
  • should we just be using varnish?
• Can I use proxy_store instead of openrestycache?
• Re-set up html minification using html-terser?
• Get blot proxy communicating with node process via unix socket

Do we eventually prune and rotate the data gathered in the various statistics cron jobs? Eventually we need to purge these we don’t want the database growing infinitely

archives and all_entries in the template engine should be bounded to a maximum of 10,000 posts

• separately we should investigate other parts of the code which perform unbounded work

Maybe horizontal scaling involves redis replicas in other aws regions?

• s3 multi region replication?

Add aws billing info to daily update

Add client status to status page (check api)

Add email warnings if no sync from client for more than x hours

---

When you re-connect to Dropbox or Google Drive, you have to re-create a folder and perform a full transfer. It would be great if we could re-use the existing folder... Maybe an option during the connect flow? Would be nice to avoid requiring a full transfer

Can the status page avoid the LRU caches somehow and fully work redis? Maybe we synthesise a route somehow? Thoughts?

hypertext template on indexes:

• title needs to link to permalink page
• anchor links in title need to link to anchor on permalink pages
• improve aesthetics and spacing of pagination links

Could we improve the performance of posts/pages with lots of images by adding loading=lazy in the output HTML?

Folder-link lookups: the rewrite tries to look up mailto: links as files in the blog's folder.

Make sync logs more useful, surface recent changes on dashboard. Group multiple files together if changed at same time, etc. make it useful!

• Add error / getHealth issues to sync log (folder.status) for Dropbox/other client Issues

Work on removing replaceFolderLinks

Excercise the backlinks with big entries / many backlinks in the benchmarks somewhere

Fix Google Drive sync outage caused by Google account suspesion

• Future fix to Google Drive client needs to never use the service account to create files (ask the user to handle the initial transfer)
• Revert 1958
• Create simple config option to disable a given client when something like the Google Drive issue happens. Make it clear on the dashboard and docs that it's not working.
• Add google drive kill switch (when platform goes down, etc.)

We need a plan for the suspension of the iCloud user as well... is the code resilient to data loss in this scenario?

• Add icloud kill switch (when platform goes down, etc.)

How do we think about a shared asset library across templates?

• e.g. shared favicons
• can we unwind some of the complete delete favion on template delete, etc. code?

Decide whether or not to revert the preset system – should it only be used for the templates index for explaining the customization features?

Test script for point-in-time restoration of specific blog folders at specific time (e.g. 2 days ago)

• test npm run restore-folder

iCloud server should periodically:

• quit safari (some error flow during auth sometimes opens it)
• close all system UI prompts (sometimes we get error for maybe accessing a revoked folder?)

Under templates index

• Clicking on file in folder pulls up post on preview site ?

Make Google Drive incremental sync efficient

• Store hash of their api responses
• Don’t touch disk
• Support huge folders with huge directories
• Start with recently changed files and dirs first

Clarify on cancel / delete / overdue pages for accounts with grandfathered subscription prices that if they re-join Blot they will lose their grandfathered rate.

Sync sweep digest (app/clients/util/syncReport.js) follow-ups

• Hourly Fix() (Dropbox, iCloud, Google Drive) runs outside the folder lock, so menu-ghosts' whole-array menu write can clobber a concurrent sync's menu change - fix in app/sync/fix/menu-ghosts for all clients (raised on PR #2001)
• Don't report a stuck lock for a blog still in its initial setup/transfer (Dropbox transfer_pending, Drive preparing, iCloud transferringToiCloud) - a big first transfer can legitimately hold the lock >1h
• Once-per-occurrence suppression treats "not seen this run" as cleared, including blogs that just weren't eligible that hour, so an ongoing error can be re-emailed after a quiet hour - remember issues until a successful walk/Fix actually clears them

Surface template choices on the templates subpages: switch from png to iframe after hovering a selection

Perform caching and header audit of site and templates

• Check logs and headers via curl
• Identify optimizations

12 factor screenshot as api

• use macserver (docker container?)

Make sure next bill amount is in all billing emails to customers

Add fast rollback action

Explain on the about page the websites started out as files in a folder. But they had to be HTML. Blot makes other file types possible too.

Get node container horizontally scalable and multi-region

• finish app/storage module with s3
• solve git storage problem

Add clear bill change info to new blog flow

Rebuild all sites to bake in folder CDN urls

Why is Ross' template so slow to install? It takes a few seconds. Let's speed it up.

Now we have many LRU caches in app/blog we need to gather data in production to rigorously determine the most effective relative sizes for the caches (include the mustache template cache in this)

• Maybe move all LRU caching into app/models and simplify app/blog?
• Maybe remove redis client side caching and instead implement LRU caching more aggressively in the models?

Improve about section

• Restructure /about into current principles and clearly dated working notes.
• Rewrite the main page to surface Blot’s purpose, independence, longevity, portability, and open source code.
• Update History, infrastructure, development tools, refunds, backups, and continuity claims.
• Date historical material such as the CDN investigation, old mistakes, and threat assessments.
• Remove outdated self-undermining statements, including dissatisfaction with the templates.
• Fix incomplete passages, spelling errors, and inconsistent use of “I” and “we.”

Fix performance bugs on various sites:

• anchor and re-enable site
• nashp (improve speed)
• jaysjay (especially archives)
  • entry pages take ~3.8s on every render, warm or cold: ~1000 Redis commands / 18MB per render while loading entries (render-probe, 2026-10-06)
• heydingus /search 0.5-0.8s warm (search_results reads ~10MB per query); its template emits …/null links that crawlers 404 on
• blot.im /questions pages 250-750ms, mostly I/O (elu 0.2-0.4), incl. 302s from /questions/:id/edit
• replaceFolderLinks looks up mailto: links as folder files (No file found in folder: mailto:…); skip non-path schemes
• using additional req trace logging and more possible LRU caching
• do we need a seperate incrementally updated index of skinny entry properties, e.g. title url etc for rendering archives / allEntries?

Implement an abstract blog storage mechanism that will simulatenously make it possible to:

• implement folder storage limits (include generated files in this amount?)
• implement s3 storage for static generated assets (and then subsequently folder files)

Fix benchmark build-corpus action issue

---

Improve Google Drive sync

• Fix bug with shared drive folders for Chuck
• Fix bug with Google Docs when disconnecting a site from Google Drive on Blot then reconnecting the site to Google Drive. We need to work out how to handle this (Google Docs are not real files)
• Improve sync efficieny with option to target a specific folder or file first (Track ids in database) and tell Chuck maybe use touch metadata method to force syncs periodically for hot files?
  • Make it possible to short-circuit the full folder traversal and immediately begin at a specific file/subfolder
• Improve html output for google docs and tell Chuck Issue:

  
  

  should be

• Fix bug with new folder setup (e.g. waiting for user to share folder on drive) which does not survive a server restart (See init.js TODO)
• Fix possible bug where it seems like sometimes the first sentence of Google Docs become titles unexpectedly – possibly a mis-identified title tag?
• How do we handle two files with same name and same parent? How does our fileId:path hash handle this? Make sure we delete the right file. Will file.title as opposed to file.name give us the deduped version?
  • Bug: move file with deduplicated name from sub folder to other folder
  • download should check for an existing file with same name but different fileId and dedupe before storing the path?
• Handle revoked credentials without tanking server
• Handle removed blog folder without error
• Handle renamed blog folder
• Handle google drive's This service is not available error
• Handle storage quota errors
• Handle 429 errors with exponential backoff retry
• Add a way for sync/build to accept a ctime as an option. We can't set this at the operating system level but google drive's API does return a createdTime value and we could use this as the true file's created date, rather than the date the file is made on the server
• Google Drive write should use the best method for large file uploading
• Add proper testing for this

Improve Dropbox sync

• Bug: removing a directory in Dropbox resync only updates the directory path; child entries wait for Fix
• Bug: Dropbox validation's catch-up sync restores last_sync from outside sync() after releasing the lock, so it can clobber a webhook sync's stamp; teach sync() not to stamp last_sync for catch-up instead
• Sync verification should instrument the sync/fix function to also report errors where the folder doesn't match the database.
• Bug: ensure that when a process dies, any blogs setting up dropbox client will be re-attempted
• Bug: Adding an additional blog to existing app folder under constant webhooks failed. New blog had correct folder but old blog still used root. To fix we need to acquire a lock on ALL blogs interacting with that dropbox account to proceed with setup process of second site.
  • Bug: handle a sync lock failure on second blog when migrating an app folder
• Bug: retry transient Dropbox API errors with exponential backoff while surfacing hard failures for manual intervention.
• Bug: re-use iCloud resolve case-conflicts function (maybe move into sync?) to ensure blogs with case-sensitive file conflicts are properly initialized on Dropbox
• Bug: Switch uploads >150 MB to the Dropbox upload session API with chunked transfer and resumable checkpoints
• Bug: run reset-to-blot when sync fails / errors
• Bug: ensure you can move the folder in full-folder permission mode
• Bug: reconnecting with the same account and same permissions should not create a new folder for full_access=true
• Bug: add a check for folder storage before doing mass upload
• Bug: handle folder full error appropriately
• Bug: add sync lock process to dropbox connection when writing folder contents
• Bug: ensure the app permission access_token is destroyed when switching to full-folder and vice versa
• Add testing for dropbox:
• Modify setup flow:
  • Phase 1: RESET FROM BLOT (NON DESTRUCTIVE, deduplicate on conflict)
  • Phase 2: RESET TO BLOT (TWICE?) then sync/FIX then begin ordinary syncing
  • Add a progress bar to the dashboard during initial client setup. Use the Google Drive and Git client progress bar implementations as reference.
• Make Dropbox sync issue emails more useful with a script to compile the logs for a single site (including macserver)
  • Adjust full_folder to add a big warning to prevent confusion about where the new folder is when transitioning permissions
  • Add a step to authentication flow which says: 'there is already a folder called 'Your blog title' in your Dropbox, would you like to use that?' then overwrite the files inside. This prevents the proliferation of confusing 'Your blog title (1)' when you re-authenticate.
• When you switch from app folder to full folder, there should be a notification indicating that your folder location has changed
• Check content hash for upload possible
• Make dropbox reset FROM abortable
• Make dropbox reset TO abortable
• Surface errors during setup on dashboard
• Surface sync issues with Dropbox client on the homepage of dashboard
• How do we make errors sticky? To make sure users click through to clients page to re-connect to Dropbox, etc...

Improve git sync

• Work out how to pipe custom error messages down to customer git clients
• Create tests for large Git repo initialization
• Make git client setup resumable / handle server restart
• Fix bug with git client which let large pushes overwhelm the main server, leading to downtime
• Handle error out of space on blot server during git client setup
• Surface errors with permissions on Git repo directory when setting up Git client
• Add sync check
• Add support for LFS and tell Jamie
• Backdate existing commits (perhaps set file ctime using info from git log?) This would sidestep the frustration experienced by Gavin when he imported an existing repository.
• Extend sync lock during long syncs
• Write test to simulate process dying and restarting mid-pull. Does the client recover? What happens if a pull fails midway? Does git recover?
• Write tests for a series of random client operations (pull, add, push, move, reset, delete) that can be seeded and reliably re-run in the case of failure.
• Move to neat promise chain api when 2.0 comes out
• Test that client must keep sending new entries and deleted entries when it recieved an error from entry.build otherwise you get ghost posts...
• send nice error message with instructions for 413 request too large: "git reset HEAD^" etc...
• Thoroughly document:
• Explain that the git client uses the master branch and repos from github will use main in the docs and on the dashboard then tell N
• Tidy up tests and reduce size and runtime
• This client should acquire a lock on the blog's folder before doing stuff to it more generally, especially during initialization etc...
• what happens if customer changes their handle? will pushing break?
  • I want them to be able to keep using their old one until someone else claims it, then I want 401 unauthorized, please reauthenticate with new username... but the remote path will be wrong?
• Rate limit authentication route?

Improve iCloud sync

• iCloud: the hourly integrity check is back; once it has run cleanly for a while, decide whether to re-enable the daily resync (resyncAllConnected) and the startup//started resync (resyncRecentlySynced), still disabled from these commits
  • 0043d91b36621ecd6aef2bcc263f51f209924792
  • 98dc9405c883825917de8e436ec620f60adaf975
• iCloud macserver should periodically validate the local blog folders to ensure the client is still connected on blot and clean up any failed initial transfers (i.e. non blog_... folders)
• iCloud: a Blot edit while the folder is missing (SOURCE_MISSING) still reaches the macserver upload, which re-creates an unshared folder (write.js, remove.js should skip the remote write)
• If we need to resolve further bugs with sync drift:
  • Implement additional parsing of modified paths from monitorer.js (brctl monitor -i) and reconcile these in addition to the fswatch module with the chokidar events for additional redundancy
• Think about adding a more efficient endpoint to sync folder state.
  • use wisdom from rsync?
  • the call to recursive list can take a way long time for large blogs (blog_3ec96xxx) so we need to increase timeout from 10 seconds to maybe multiple minutes...
    • can we get rid of recursive list entirely if we simply call brctl download on the paths flagged by brctl monitor?
  • don't hold sync lock the whole time?
  • fetch file list, compute changes, then lock, then apply?
    • we'll need to update the call sites since the lock will be handled within
• Daily icloud resync should also log how long the longest folder took to resync.
• Add stats on response time to iCloud server health check logging
  • maybe monitor response time average and email notify when it increases dramatically?
• Smooth over setup process, add guidance for 'mode' of share panel (e.g. copy vs. collaborate) and maybe a screenshot
• Add a progress bar to the dashboard during initial client setup. Use the Google Drive and Git client progress bar implementations as reference.
• Make iCloud sync issue emails more useful with a script to compile the logs for a single site (including macserver)
  • Add logging instrumentation of modified file paths to sync verification to aid in analysis of sync drift
• Enforce a configurable timeout and cancellation for build jobs triggered by iCloud sync events to avoid the concurrent sync bug referenced in sync/set.js.
• Start writing tests for icloud client
• Start writing tests for macserver
• Instead of our bespoke header modifiedTime we should use 'Last-Modified' and note we don't need to set this for route using res.download – res.download does this automatically
• Fix bug: low priority, this just causes an error in the logs when freeing up disk space. Problem: attempting to evict a file within a .pages directory (a special apple file) e.g. Post.pages/preview.jpg will fail: you need to evict the parent directory perhaps?
• Remove evict route
• Rename /watch route to setup-complete
• Remove recursiveReaddirRoute
  • instead, to sync folder tree, whenever readddir is called, it should maintain an internal map of the last time a blog's subfolder tree was recursively checked, and call it? or maybe just call ls and brctl download on the specific folder? or is this too slow?
• Investigate the EHOSTUNREACH erors on the main server when attempting to communicate with the mac server – some firewall issue with mac hosting provider?
• Investigate: can we modify app/clients/icloud/routes/site/upload.js and app/clients/icloud/macserver/httpClient/upload.js to exchange a content hash? to prevent needless re-upload?
• Investigate setting up paralell server with 'Advanced Data Protection' enabled so users with this setting can sync to Blot as well

---

Fix bugs with templates

• Fix responsive thumbnail component on photo template (it loads large thumbnail every time)
• Fix 'back' button in photo template and tell Bibi
• Fix side navigation on Blog template and tell Sava
• Add layout fix to blog template and tell Lewis
• Fix bug with header position on the blog template
• Make 'close' button on entry permalink page of reference template work like the photo template and tell Marco
• Fix bugs with documentation template and tell Ray
• Fix bug with Documentation template and the setting 'Adds zoom to large images' and tell customer
• Improve post navigation on Portfolio template within tagged collections and tell Questioner
• Links template: Navigation:
  • Add dropdown support to navigation Entry page:
  • Close button should go 'back'
  • Add layout CSS
  • Next / prev should respect tag collections
  • Add exif info
  • Add backlinks

Fix bug with placeholder files when switching clients which will lead to data loss because Blot will not transfer the contents of placeholder files across

Fix bug with the paypal next bill dates are not syncing correctly for some users (e.g.slopserve)

Fix bug with CDN replaceFolder not working with relative path to CSS file on wall.blot.im

Fix bug with huge draft crashing the sync server

• /a327ex/drafts/it_follows.txt
• Chunk large drafts into multiple messages to avoid memory issues?

Fix bug with overdue subscription payment form not working

• Possible bug introduced by recent stripe upgrade
• Migrate Stripe library to v7

Fix bugs with custom domain setup

• Build in some of these features from the demo: https://customdomain.ai/
• Validate the domain is pointing to Blot before allowing resty-auto-ssl to request a cert. we seem to get into a DNS mismatch where a client thinks a domain points to Blot, and triggers a cert request through resty-auto-ssl but whatever DNS server Blot's server points to has the stale record (pointing elsewhere)
• Disable redirect to HTTPS for blogs by default (to ensure old browsers without SNI work) but ensure all prominent links to the blog on dashboard use HTTPS
• cache domain validity and display checkmark on dashboard

Fix bug with link format using filename

• follow up with yct

Fix bug with light mode colors

• tell Gary

Fix bug with menu which occur when multiple pages are added at once Menu needs concurrency or atomic operations (so we can build posts in parallel)

• move menu into its own model (it's possible to add thousands of entries to it...)

Fix bug with many twitter embeds

• and tell Nash

Fix bug with filtering paginated list of entries

• tell Roy and Josh and Logan

Fix bugs with template engine:

• Prevent double submission on template editor at insertion and front-end levels
• Make it possible to rename template directories and tell Darren and tell questioner and questioner
• Investigate bug with template views whose names include hashtags and follow up with Jay
• Fix bug with undeleteable template files prefixed with dot and tell Stuart
• Fix bug with template editor and ensure POST error messages are passed to client properly through ajax
  • to trigger, submit a >5mb file on web template editor
• Fix bug with chinese characters in template name and follow up with Alex
• Fix bug with date settings on the template editor for magazine template
• Fix date selector for template and tell Zach
• Fix bug with forking / editing new templates and tell Jarrod when local template folders better reflect template names

Fix bug with wordpress importer

• follow up with Simon

Fix bug with date metadata on Vincent's blog and follow up. I suspect it's to do with the ordering of date tokens.

Fix bug with potentially TeX-related escaping issues and tell Anthony and Roy

Fix issue with variable passed to dateFormat function and tell Leo

Fix bug with handling of queries and tell this person.

Fix bug with tags that contain slashes:

• tell Jack
• tell Dave

Fix bug with search for 'Raycast' and tell Anders

Fix bug and tell Vanessa which injects <p> tags

Fix bug with blockquotes and tell Kerim. Multiple ">" symbols on the blank paragraphs between other quoted text are not formatting properly.

Fix bug with indented code in pre tags and tell Michael

Fix bug with nested Markdown and tell Martin:

Fix bug with Client.write which does not propogate sync errors... to demonstrate this, revoke dropbox token then try to enable local editing.

Fix bug with Google Drive client and Stephen's large blog

• It also seems like server crashes when a massive google drive folder is synced, with a directory containing 6772 items

Fix bug accessibility issue with menu links reordering and tell Grace

Fix bug with accessibility issues with scrolling and code blocks and tell Eli if I make any changes

---

Record new video

Make it possible to create a site for free then require subscription to make it public

---

Follow-ups from drag-and-drop template upload:

• Move the folder browser (views/dashboard/folder/directory.html) onto the shared views/js/collect-dropped-files.js instead of its inline copy of the same traversal
• Surface template metadata.errors in the dashboard editor, not only on the preview subdomain, so a template with a broken view says so where it is edited
• Reconcile the two url defaults for a new view: source-code.js maps foo.html to /foo, readFromFolder and the upload route map it to /foo.html

New templates:

• Release Fede's theme as profile template
• Transfer theme for John

Switch templates from entries.html to index.html with explicit url: ['/', '/page/:page'].

• fix docs on routing order

Improve markdown:

• Turn on 'preserve linebreaks by default' for markdown
• Extend markdown parser to support content blocks syntax
• Extend markdown parser to add a special class to a list of only images for targeting into an image grid or gallery?
  • - ![Image](_image.jpg)... -> <ul class="image-list">... could then be used in CSS?

Improve webloc bookmarks:

• If the bookmark file points to a google doc, can we support google docs even from Dropbox? if the doc is public?

Improve images:

• Use avif/webp in responsive tags with jpg fallback for images in posts
  • use webp for image post conversion format app/build/converters/img/index.js
  • use for image caching module
  • add finer-control options for image caching / quality compression
    • Fetch dimensions of images even if optimize and cache is disable, so that image zoom can work. Tell Joshua
  • use for thumbnails
  • use for responsive images in posts / generated from image files
  • Tell Audun
  • Tell Robin
  • Implement for thumbnails too
• Add support for large image tiling and zooming with photoswipe or add a plugin using OpenSeadragon to add Deep Zoom capabilities for huge megapixel images so you could publish them directly to Blot.
• Extract multiple thumbnails per blog post (as many as there are images) for cooler image grid possibilities. Add a property to entries which lets you list all the images in a post so Jeancan add them to his sitemap
• Add size / time limits to image caching function which fetches remote images
• Image cacher should work for webp, tiff, etc. as well as JPG

Improvements to docs and documentation:

• Add way in to the updates page on the index page (i.e. surface recent tasks done and to do)
• Add way in to the questions page on the index page (i.e. surface recent questions)
• Rewrite code which generates a fake 'macOS' folder to work cross platform:
  • if the user is browsing from Windows create a windows folder
  • if the user is browsing from linux create a linux folder
  • Windows video + windows finder.css
• Personalize the documentation for logged in customers
• Improve tools section of docs
  • Update obsidian tools pages (improve)
  • Improve tools/bear
    • Reach out to Bear to let them know about textbundle support
  • Improve tools/ulysses
    • Reach out to Ulysses to let them know about textbundle support
  • Create tools pages for AI assistants (Claude Code, Cursor, Codex) explaining how to use them to create templates

Improvements to template editor:

• Do preview subdomains respond to etags?
• Make preview hot reload more efficient by only reloading when the tab is visible
  • the injected script in app/blog/render/middleware.js and app/blog/views/template-error.html does new EventSource('/__blot/preview/reload').onmessage = function() { window.location.reload(); } on every message, including background tabs / hidden template-editor iframes. the expensive part is the full render that location.reload() triggers, not the SSE connection (keep that open; 10s heartbeats are cheap)
  • follow the visibility pattern in app/views/js/sync_status.js: if document.visibilityState === "hidden", set a pending flag; on visibilitychange to visible, reload once
  • debounce? no long debounce. the server already publishes once per completed sync that had changes (app/sync/index.js after the cacheID bump), so events are not per-keystroke. coalesce instead: one pending reload is enough if several events arrive while hidden or while a reload is already in flight. a short (100–300ms) debounce would only matter for back-to-back completed syncs while the tab is visible
  • etags? not nicely, not yet. location.reload() may send If-None-Match / If-Modified-Since and openresty already forwards those (proxy/config/reverse-proxy-preview.conf). render middleware sets Last-Modified from cacheID and Express default weak etags come from the body, but freshness is checked in res.send() after the expensive render, so a 304 still costs a full render. skipping the reload when hidden is the real win. a cheap cacheID-based etag checked before render would actually leverage them — related to the etag question above
• Add commit mono font to font picker
• File upload feature (_url) needs to generate image dimensions too.. maybe it needs MORE power
• Enforce size limits on setting tempalte views
• Allow text entry fields in template settings from the dashboard for template locals, and tell Omar (Build markdown description feature into template editor. Support default site description)
• Introduce collapsible sections or a separately scrollable div on the template dashboard settings, and tell Omar
• Build image upload feature into template editor. Support default banner images and image avatars, with default images set, maybe some kind of global library feature across templates...
• Build favicon generator into template editor. Make it distinct from avatar. Generate proper favicons from uploaded avatars (to prevent streched favicons when non-square avatars are uploaded)
• I wonder if we can Can we avoid storing font styles in package.json, just the ID? then inject these at render time?

Improvements to templates:

• Improve hypertext template to support listing pages as well as posts in the sidebar
• Add media-query to prevent typeset hanging punctuation on small screens (it looks weird) and tell Shibel
• Improve sidebar blog template navigation on small screens

Improve importer:

• Add way to restore a blog from export
• Update question on what to do to import posts from wordpress or squarespace: instead of contact me, reference the importer.
• Can we offload an import into a resource constrained child process?
• Can we queue up imports and process them one at a time?
• Add interface to move import into folder directly
• Add scheduled imports on regular basis into folder directly
• Add nice import step under 'Services' and as part of sign up
• Improve Ghost importer and tell Vin
• Improve Are.na importer so it handles descriptions + also other file types (e.g. PDF, text) and tell Marco
• Add micro.blog import to dashboard and tell [Gabriel](https://mail.google.com/mail/u/0/#inbox/FMfcgzGwJRvfgrRhXwqDNDlxbpXmRlbd]
• Create importer script for John's posts from flickr
• Create importer script for tumblr text files and tell Sawyer
• Create importer script for Weebly and tell David
• Map excerpt metadata to summary for Jekyll converter importer
• Look into how Hexo has written imports for blogger, joomla, rss, wordpress.
• Add an importer for Twitter and sell Sawyer

Let customers duplicate an existing site as a starting point

Improve exporter:

• Make it possible to configure the export file format (e.g. instead of markdown, word documents) on the importer
• Add export button to overdue subscription page

Blogs have huge numbers of archived templates – modify the tempalte archiver to only 'create' templates for sites using them, otherwise disappear them. work out how to clean up some of the many archived templates added to sites... (remove ones with zero changes?)

Remove the legacy redis hash routing from cdn/index.js and key definition

• also create new script to forcefully migrate the dangling hashes from redis to disk, if they exist

Fix script to replace cssURL to handle blogs with invalid domains (it's breaking the migration)

Replace existing cssURL and scriptURL and css_url and script_url properties in templates with new cdn helper

• do this by editing the templates
• once confirmed all templates are fixed, remove the code and documentation for these variables

Replace existing asset helper properties in templates with new cdn helper

• Check it's only used for CSS/JS...

Can we optimize renderView in some way?

• use lru cache or moize libraries? back against cacheID property of blog?
• Memoize load template / load full view / load blog... others?
• use a proper library, cap memory, etc.
• should majorly improve performance on popular sites

---

Implement TOC Plugin

• Create a plugin-based TOC system that works for all file types. The plugin will generate TOC HTML from headings and return it via the result object mechanism from task 1.
• Tell Omar

Investigate replybox for comments

Add checkbox to toggle newsletter subscription to account page

Make it possible to connect two colors in the template editor, e.g. navigation background color -> same as background color

Add lang selector (map english -> en) to template editor

Create a new template local '{{{system-css}}}' which merges {{{appCSS}}}, {{{font.styles}}}, {{{title_font.styles}}}

Secure deploy ssh on GitHub

Add GitHub deploy button for macserver

Add GitHub deploy button for openresty

Set up AWS cost anomaly detection

• add daily aws cost to email update

---

Add support for 2FA (Two factor authentication) then tell Luke, Max, Chaz, Prathamesh and Tom.

Add support for SSO then tell Prathamesh

Improve the camera make and model names extracted by exiftool using a third-party library, e.g. SONY ILCE-7M3 -> Sony α7 III

• Tell Chuck

Compute 'words' text property for each entry (unique words, sorted possibly?) to improve the speed of search queries

Identify areas of the codebase which need resource limits to be more secure, e.g. max file size for the image converter, max file size for the docx converter. look across the codebase and create a series of implementation tasks for the most pressing

Modify slowest function to aggregate by upstream response time

• amend daily update:
• send daily report of slowest URLs
• send daily report of biggest URLs and bandwidth

Build out simple discover page with links to publish in daily email

Add photo/pictures/pics/images grid to tagged page of blog template and photos subfolder to default folder.

Add plugin to generate anchor links next to headings automatically and tell Peter

Refactor front end domain page to have radio button to seelct between custom domain and blot subdomain Inbox

• don't delete custom domain, though (maybe add property redirectDomain ?)

Think about how to improve the efficieny of the app/views/js/sync_status.js response from the server when loading the new contents of .live-updates. perhaps we could intercept send requests and use cheerio to filter out everything except the live update div? or some other method? and once we do that we could then leverage etags for improved performance too

Add screenshot feature to macserver:

Reach out to include Blot on textbundle website

Look into figma make

Can we use lightningcss to replace the existing css minifier for performance?

Transformer should short-circuit network lookups to {{config.host}} or {{cdn}} urls and pull the files directly

• and have limits on resource size and time

Add support for Apple Pay and Google Pay by upgrading Stripe API version we're using

Improve performance rendering sitemaps by using sitemap indexes, and sharding the sitemap.

• requires adding pagination for listing pages
• requires using the url / param feature, e.g. [sitemap.xml, sitemap.posts.:page.xml, sitemap.pages.:page.xml]

Speed up 404s

• Analyze req trace
• Remove case insensitive lookup and add path resolution to plugins

Improvements to template engine:

• Speed up global template build by not even invoke the setView function if there are no changes... (this avoids running updateCDNManifest when SITE template has no changes)
• Implement limits for allentries, sitemap, etc.
• Maybe we mix cacheID into query string for {{#cdn}} on preview subdomains with caching headers for improved template editor performance without expensive hash computation?
• Performance improvements
  • Limit number of posts in rss feed to 15 (modify recent entries?)
  • Add pagination to /tagged
    • should resolve issue with https://eli.li/tagged/og
• Improve various performance issues with archives page and sitemap on large sites - maybe allEntries only retrieves some properties and paginates through rather than a single request?
  • highlighted by https://jimhardy-datascience.com/search?q=2026 and https://www.nitinpai.in/archives
  • tried storing entries as a Redis hash + narrowing reads to just the fields a template references (PR #1831 unwinds it) - regressed prod performance and did not help either site. Root cause on these two sites was never diagnosed; don't retry the same approach without figuring that out first.
• why is the feed so huge? are we compressing feeds? so slow for nginx?
• finish function to render files in the user's folder dynamically and follow up with questioner
• maybe write tool to check before/after for lots of test sites? real life sites on blot? error if diff, or offer chance to test?
• Add a related entry list and tell David and Thomas and questioner and questioner
• Consider adding support for nested page structure - generate nested menu items from nested folders in /pages
• Switch from mustache to handlebars
  • Precompile using template content hash
  • Add strict verification for output differences
  • Can we generally do a ‘static’ build out directly into a cache (e.g. openresty)?
• Make it possible to query recent entries by tag and tell Peter
• Assign guid template IDs
• Assign unique path to template
• Assign template withs with full filename
• Make writeToFolder more efficient by not removing the entire folder each time (only remove orphaned files) and by no rewriting identical files
• Add search bar to template editor
• Make syntax highlight dark mode light mode compatible somehow?
• Work out why rebuilding template from folder clears cache even if the template is not changed
  • stop this
• Make it possible to source a template from a github repo (live) and tell Nash
• Add new template lambdas
  • Whitespace trimming and tell Chad
  • Number formatting (e.g. add commas to int.) and tell Chad
• Make template slug seperate from the template ID, simply for human readability
• Investigate performance of rich text editor in Safari on template editor and follow up with Rachel
• Add blog.statistics and entry.statistics (total words, etc.) and tell Pratik
• Work out how to make live but saveable modifications for preview purposes on the template editor. I have already established a messaging channel between preview HTML pages and the template editor with the script injected in render/middleware.
• Add background patterns to template editor tools, tell Ish
• Add 'group by' function which would allow you to group posts by month/ year in the same way that {{#archives}} lets you and tell Jay
• Respond to Michael's thoughtful emails about the template system
• Add support for chronological template ordering and follow up with questions
• Add way to iterate over public and private pages to allow sitemap to work properly
  • tell Rodrigo
  • questioner
  • Tell Steve
  • Jack
• Allow the name/slug of a template to change and tell Marcus and Pratik and Brad when the name of a locally-edited template in your blog's folder updates
• Expose request query to views and tell Abhishek
• Add way to query list of posts using SQL?
  • Add support for entry lists of variable length and tell Logan
• Add support for the setting of custom headers (e.g. to enable CORS) and tell Roy
• Add support for listing files in arbritary folders (e.g. use a custom folder as a 'public' folder) and tell Nash and Marco
• Add way to list directory contents and tell Diego
• Move to data directory for templates
  • Serve the template directory as a static folder, allowing things like favicon.png and humans.txt to not become blog posts. Follow up with Jamie
• Re-do url redirector to use route-parser and properly support regular expressions
• Add support for paginated list of entries on other pages and Dan tell Kevin and Eli and Sven and Nikhil and Adam and Madino and Ryan
• Add dynamic querying of entry list and tell Naoki
• Add a popular entry list
• Add way to auto-refresh page during template development on preview subdomains. Tell Ryan
• Add way to list files at a path and tell Matt and Daisy
• General:
  • Add paginateable entries list
  • Add downloadable sites
  • Add static file uploader
  • Make templates folder based?
    • data/templates/blog_id/template_name
  • Add webfont uploader
  • Generate favicons of different sizes from uploaded image
• Re-enable partial templates in posts/pages
  • Tell Matt
• Add pagination to archives and tagged pages on default templates using a new tag
• Inject title metadata into h1 tag for all the text-based default templates and tell questioners
• Add 'download as zip' option to template editor
• Create starter kits for each template
  • e.g. a starter kit on the blog template for a writer, or a photographer, or a journalist, or an academic.
  • these could be downloaded as folders on the template page?
  • Get David's feedback when one exists for writers
• Create an index of customer-created templates and add Matt's theme
• Add 'Hidden: yes' metadata to replace 'Menu: no' and tell Sebastien
• Add method to merge updates into custom templates and tell Dave
• Redirect URLs with trailing slash when possible (i.e. not directory browsing) to fix possible duplicate content penalties and tell Marius
  • Add rel=canonical to all the default templates, or resolve the problem that Blot serves the same thing to URLs with a trailing slash, and without.
  • If we do add rel=canonical, make sure to consider the situation where someone mounts a page to the index of their site, e.g. /page/1 and / will now be different.
• add support for dynamic partials to make collections possible, e.g. {{> /:tag.html}} ?
• Add neat interface to customize blog level variables?
• How do we transition from a global date display format to one that templates have control over?
• Consider making file-based template directory a git repo and using this to implement basic undo-redo functionality.
• Disable re-rendering for locals
• Add better customization options
• Local template editing does not clear cache
• Use Hogan for precompiling and rendering templates for performance?
• Parse tokens and params from routes and tell Jason
• Add way to allow all entry lists to access {{#months}} sorting options just like the archives page.
• Tell Marco when arbirary list sorting is ready
• Add support for fetching only some properties of an entry (ideally only those that are used). prevent some properties from being fetched in some lists (html, for example in all_entries). Would be a good candidate for hgetall or hget?

Launch new server and test that the new scripts to mount the instance store and data ebs-disk work

Automate the detection and removal of long-overdue sites (> 1 year)

• Prevent long-overdue customers from accessing the service, then tell Matvey

Implement toString for all existing template lambda, see retrieve/updated.js for an example

• Map {{date}} => {{#date}}...{{/date}} and {{#updated}} and {{#created}} etc.
• Update developer docs
• Can we use this technique elsewhere? Simple string vs. function...

Improve folder to CDN pipeline

• Could we respond with redirects to Blot's CDN?
• Why isn't the WAV file in this footnote being properly replaced with a CDN URL?
  • it's because it contains URL-encoded chars – replace
  • https://diwank.space/field-notes-from-shipping-real-code-with-claude
• Why isn't this file?
  • https://yangrobotics.com/posts/peng-2.md
  • possibly because it's a fully-qualified URL? we need to handle these I think
• Maybe all 'asset requests' should return 302? even 301 if the url is versioned?

Add per request resource monitoring, e.g. heap size before/after

Move image zoom plugin to templates

• Fix bug with image zoomer on mobile devices and tell Alessandro
• Fix bug with images zooming inside footnotes and tell Nathan
• Document image zoom disable feature ?zoom=false and tell questioner For each blog: if image zoom plugin enabled: for each template: inject required CSS just above {{{appCSS}}} and save inject required JS just above {{{appJS}}} and save disable image zoom plugin Remove image zoom plugin

Reduce service provider risk:

• move domain registrar from AWS to Epik?
• Remove dependence on AWS
  • Adds offsite (i.e. non AWS backups) backups
  • Write out plan to move off AWS
    • DNS
    • EC2
    • S3 for backups
    • Cloudfront for CDN
    • backup blogs folder + db somewhere offsite at least once per week
• We should have a plan to switch CDN from Bunny to Cloudfront in an emergency
• Should be a simple DNS change
• write plan to transition off Bunny if needed
  • ideally all this should involved is changing some DNS records
• test plan to transition off Bunny
• write plan to transition off AWS if needed
  • ideally all this should involved is changing some DNS records
• test plan to transition off AWS

Implement mac server screenshot endpoint

• We should use the helper/transformer to speed up screenshot re-generation

Update redis client for improved performance

• perhaps to ioredis?

Update redis server to version 8

• take advantage of full text search? or switch to postgres?

Move folder lock location to instance ssd for improved reliability and performance (non-networked drive...) – could be inside the cache

Extend ignore files feature (currently files starting with _ or folders starting with _)

• tell Troy
• Follow up with Shibel about how to ignore - make it possible to achieve through a global 'ignore' setting on services page of dashboard?
• could ignore all '.jpg' say, or all 'index.html' files, etc.
• Can we add a 'folder rules' section to create ignore globs, multi post globs, etc. arbritrary paths and keep clean URLS (no underscores). Sitewide .blotignore file?
• does rebuild update ignored files list too?
• handle new pathNormalizer behaviour
• 'ignoredFilesKey'
• add 'ignored ghosts' check to sync/fix
• Add Ignore: yes metadata to trigger the ignored file behaviour

Think about how to purge the old versions of files cached by bunny's permacache

Reduce size of css file by only shipping katex css once the site first publishes a post with katex

Testing:

• Write unit tests for database models which have no tests subdirectory
  • e.g. models/404, models/entries, models/redirects
• Add tests for webp thumbnails
• Add integration tests for cdn paths with weird view names, etc.
• Add integration tests for preview subdomain logic (skip for my- not for site)
• Add end-to-end tests for brochure site to test sign up flow (use Playwright? Phantom? Chrome headless)
• Add spell checking test
• Write browser based integration tests for core functionality
• how can we test the 'reload-server' script?
• how can we test the 'scheduler' functions? we ran into error where it was running twice

Move from EBS to s3

• Create abstract models/storage that could be used to eventually migrate blog data folders onto s3:
  • 1. Add app/storage/index.js that exposes namespaced helpers (e.g., storage.blogs.resolve(blogID, relativePath), storage.blogs.readFile, storage.static.writeFile) implemented with fs-extra and the existing config.blog_folder_dir / config.blog_static_files_dir.
• Lower costs using s3 instead of EBS disks for static file storage
• Goal is cost reduction and improved scalability
  • should lower storage costs + transfer from $200/month to $20/month
  • no more charges for unused GB disk space
  • lower cost of S3 vs EBS
  • no issue of 'running out of space' which has taken down service multiple times
• General considerations:
  • what s3 category to use? standard? 1 zone express? latency is important for some use-cases, e.g. blog folder
  • the static files (e.g. thumbnails) are more long term infrequent access though (because of CDN)
  • do we re-organize how files are stored? one bucket per site?
  • make sure s3 versioning is disabled?
  • how to back up? maybe have a special bucket for blog folder data? everything else is downstream of the folder
• Come up with plan for static files directory
  • rewrite docx, odt, clients to use a temp dir, then
  • create an abstract helper function which can be used to write files to static directory
    • make this file also upload to s3
    • write script to sync existing static files to s3
  • we probably don't need a caching layer for static files since static files will end up in CDN storage after first request
• Come up with plan for blog files directory
• abstract write / read from blog directory
• we probably need a caching layer using instance SSD as LRU cache of say 10GB
• when we rebuild all files on a site, we'll probably want to download all the folder? or maybe just have a SSD LRU caching layer built in that can adapt.
• Rewrite git client as follows:
  • Store bare repos (e.g. blog.git) on s3 as source of truth
  • On push, fetch bare repo from s3 and store it on instance SSD in 'cache'
  • After push, resync bare repo to s3
  • Compare diff on bare repo to identify changed files
  • For new or modified files, check out those files only to a tmp folder and then copy across to blog folder dir directly and trigger folder.update
  • Remove the live repo from the blog folder dir
  • Other ideas which I decided against:
    • I thought about using an s3fs but it seems risky and likely too slow
• Once this is done, the nginx + node server is 'stateless' and could be replicated easily across aws regions
• We could combine this with the containerization of openresty

Resolve bad links (e.g. incorrect case) in files at build-time rather than run-time so we can avoid the expensive disk lookups in blog/assets.js

Add a highlevel function which clients can expose to open a file (e.g. link to dropbox.com or google drive)?

Run image analysis on all images in post

• extract colors for searching
• use something to identify related images

Improvements to questions section:

• Add 'blue dot' notifications icon to updates and questions based on last post
• Add colors to tags on questions section:
  • https://gauthierroussilhe.com/articles
  • https://designportland.org/
• Add a filter for topic index to show questions by: 'Newest', 'Active', 'Unanswered', 'Votes'

Add auto template translator

Add global site search of:

• files on site
• posts on site
• documentation
• questions
• templates

Add an option to use the date modified property of the file, rather than the date created property as the publish date. This would be especially useful for the Dropbox client, which nukes the original file creation date but preserves the client modified data. This should be opt in because it could lead to misleading behaviour where an old file is dragged into your folder and is published way deep in your archives. Tell Dirt

Improvements to local folder experience

• Interpolate fonts based on an ID (a la web template editor)
• Interpolate syntax highlighting based on an ID

Surface recently edited posts on the sites tab of the dashboard

Can we support white-label cdn subdomain so there are no Blot CDN links?

Write tests for animated gif thumbnail

Surface Blot client sync errors on dashboard

• add email notifications for major sync errors (i.e. dropbox token revoked)
• surface dropbox out of space error
• Expose markdown build errors on the dashboard to prevent weird non-update situations where it looks like the sync is stuck

---

Preserve single line breaks in Word Docs for poetry and tell Isabel and tell Paul

Think about migrating to s3 for blog data storage

• Purge thumbnails and cached images which are no longer in use
• Abstract methods which read, write and readdir the blog folder? static files folder?
• What about pointing the CDN to an s3 bucket then redirecting to upstream for 404?

For broken embedded images, attempt to resolve based on a shortest path approach, as in, traverse the folder and attempt to find an image with a matching name

• Tell Ellane

---

Embed feature

• Add support for native PDF embeds in posts, just put a link to a PDF or a path, e.g. /_assets/file.pdf --> would become a nice embed.
• Make it possible to embed PDF like so ![Video](...)
• Make it possible to embed video like so ![Video](...)
• Add support for native MP3 embeds in posts, just put a link to an MP3 or a path, e.g. /_assets/file.mp3 --> would become a nice embed then tell Luke and Nash and Andy and Charlie and Konstantin
• Add soundcloud embeds then tell Jonas, Nashp, and Anders and Richard
• Add are.na embeds and tell Marco

Improvements to search engine:

• Make search use 'AND' instead of 'OR' for multiple terms
• Remove drafts and tell Paul
• Make search: true work and tell Paul

Rewrite broken link tests

Add pause subscription for Stripe and PayPal

• Enable hibernation of blogs

Add disclaimer to pricing page for old subscribers

Add Generative tree for account page

Add {{#posts-without-:tag}} and tell steve

---

Add aws cost usage and forecast to daily update email

• https://docs.aws.amazon.com/cli/latest/reference/ce/get-cost-and-usage.html

Ensure that overdue PayPal subscription webhooks map to notification emails

Clean up CDN permanent storage:

• Purge old docs CSS and JS from CDN storage cache
• Purge old {{#cdn}} helper responses from /template/ route for blogs

Make sure main server can hard restart without issue

Data use minimization:

• Write automated script to archive overdue blogs from disk and db
• Write script to remove unused and unaccessed static files from within blog folders
  • we'll also need to purge the cache of the transformer module for the file

When a post is deleted, we should remove everything except the created time?

Create a queue for purging cache, group requests together to prevent stampede

Prevent overzealous purging of cache for syncs in which the folder does not change

Add WebDav support

Improve paypal integration:

• Get create new blog working
  • Add way to modify subscription quantity for paypal users
• Create new router for paypal subscription settings
• Test subscription with real paypal account
  • Test new blog creation
  • Test new blog deletion
  • Test switch to annual billing
  • Test account deletion
• Make it possible to add paypal as a payment method for existing stripe users
• Verify signature integrity of paypal webhooks
• Use PayPal buttons api to modify subscription properties
• Get account page working
  • Add way to cancel subscription from dashboard
  • Add way to delete account from dashboard
• Update scheduleEmailNotification for paypal customers

Make {{active}} strip the domain from the link before checking? Or automatically strip the domain from internal links on Blot's dashboard?

Store path -> md5 hash on sync

• Use to auto version links to files in your folder so they can be cached

---

Adjust auto titles of 'screenshot at 12:22pm' and make sure the date parser can handle them

Adjust title generator so h2 titles within the file dont become the title of the post – this is helpful for obsidian

---

Server improvements:

• Copy the cron tab into the new server setup script
• Add more stringent checks to scripts which mount instance storage and external volumes
• Understand why the server gets so slow when we switch from SSD ebs to HDD ebs
• it would be ideal to use HDD for bulk of blog storage, and take advantage of the local SSD for high performance operations
  • Investigate use of aws s3 sync for backing up ebs volume rather than snapshots
• Test reboot for all servers

Add support for proxying particular analytics files and tell questioner

One bad latex equation should not tank the build process for the whole post

Add support for activitypub and tell

• Tim
• Questioner
• Rhys

Methods to improve server performance

• consider expanding the replication of the perma_cache across more regions
• consider switching to more expensive CDN type (i.e. not volume)
• switch to more expensive EBS for disk of server

---

Add support for paying with crypto, BTC ETH

• Consider https://github.com/btcpayserver https://github.com/btcpayserver/node-btcpay
• Tell Chaz
• Tell Anita
• Tell Arc
• Tell Dmitriy

Improvements to server

• test maintenance and redirector on dashboard
• restrict post to /dashboard route so we can tune nginx logic?
• tune performance of express-mustache in production and development
• add optional HTML/JS/CSS minification to express-disk-cache
• Test overdue subscription re-start
• Prevent GET request to log in page from setting connect.sid cookie
  • or otherwise work out how to try and cache everything at nginx
  • log in route should clear 'logged_in_to_blot' cookie if not logged in
• Use Lory to add a slider to the homepage

Improvements to sync/reset scripts:

• reset should produce a report which I get email

---

Add profile photo cropping/masking tool

• include circle, oval, and other shapes (e.g. https://www.jayceeday.com)

Add way to escape commas in tags and tell questioner

---

Add simple form endpoint, could be used for contact forms

Add theme with photoswipe and use sharp's tile generation feature along with

Add way to set the slug-part of a URL for a post and tell George and follow up for post in Questions and tell Jarrod

---

Investigate migrating from Redis to SQLite per-blog database

• Evaluate feasibility of storing one SQLite database per blog in data/blogs/{blogID}/blog.db
• Assess performance implications (memory usage, query speed, connection management)

Can Blot blogs be federated? Respond to Nigel once I do some research on this

---

Support larger files on DB client and tell Marco

---

Add tag to iterate over all images in a folder and tell Laurel

---

Make it possible to link between org mode files with: [[file:another-test.org][another test]] and tell questioners

Improve backlink's tolerance

• Add support for full qualified URLs in the backlinks list for each entry. For example, if your blog is example.com, the following link will not appear as an internal link: Foo because build/prepare/internalLinks is not aware of the blog domain/handle
  • Tell Eli
• Add support for internal links without a leading slash
  • Tell Laurel

---

Move citations out of {{{body}}} and into a new tag {{{citations}}} and tell questioner and Mikka? maybe more structured, so you could also do: {{#citations}}...{{/citations}}?

---

Improvements to teaser generator

• Improve estimate of teaser length
• Remove trailing non-text elements (like

  ---

  )
• Add {{small}} and {{long}} versions of teaser?
• Internal links (e.g. #section) in {{{teaser}}} should be resolved against the entry's permalink.
• Add hasBreakpoint property to entry to allow greater control over display of {{{teaser}}} and tell Pratik
• Make it possible for users to select length of teasers and Leo

Make it possible to disable draft preview files

Performance improvements

• Faster server, with SSL termination across CDN
• Use the --trace-sync-io command-line flag to determine if Blot is using any sync methods inappropriately
• Can we keep the pandoc child_process running and pipe stuff in and out for speed? Measure before after to ensure this helps
• Add request level timeouts to prevent server swamping
  • ensure to remove the timeout from the SSE streaming routes (drafts, sync status on dashboard)
  • read this guide: http://expressjs.com/en/resources/middleware/timeout.html
  • it seems we also need to cancel the resources involved in responding to the request (i.e. disk io, db calls) for this to be really useful
  • Add a timeout of 15 seconds for building posts.

Add line numbers to highlight.js hljs code tags

Convert Vitepress to Blot and tell Nicolas

Expose log files on blog's dashboard and tell Roman, Sönke and questioner and John

• Wikilinks should have a class property of 'wikilink' rather than a title attribute and tell questioner
• Improve wikilinks lookup algorithm:
  • Add support for 'shortest path where possible' on Obsidian
    • search current directory for filename
    • search subdirectories for filename
    • search parent directories for filename
  • Add support for 'absolute path' on Obsidian when the vault is a sub directory of your blog folder
    • How best to accomplish this?
  • Add support for deleted files (e.g. [[Target]] -> target.txt still works if target.txt is renamed to foo.txt)

---

Improvements to metadata extractor:

• Adjust summary generator such that a subtitle is used for summary text (i.e. an h2 directly following an h1) We still want to prevent the scenario in which a later h2 becomes the summary, though. Notify Jessica and rebuild Jessica's site
• Add support for entry property with escaped double quotes, tell Jay

Improve questions section of site, in approximate order of importance:

• Add way to mass-rename tags
• Add display aliases for tags e.g. how -> How to use Blot, etc.
• Store version history of the question and answer edits
• give the 'ask' and 'answer' textarea inputs the full functionality of the GitHub issue/PR comment inputs (i.e. code snippets / preview / copy/paste screenshots from clipboard)
• email notifications (There's an existing system in place for sending emails in app/helper/email)
• upvoting for topics and upvote-based ranking on the topic index
• show vote count and answer count next to each question on index
• 'run code snippet' feature / similar to trinkets.io

Rewrite transformer (in app/helper/transformer)

• Speak HTTP properly, respect backoffs, use/send eTags
• Share information across blogs?
• Improve transformer so it checks a file's MTIME against a stored value before rehashing the file?
• Adjust interface to accept a number of directories. Make it 'unaware' of blog IDs?
• Add method which accepts a path or URL and flushes what is stored for it, update the rebuild entry script
• Cache errors. When a post contains multiple identical links to images that timeout/404, the thumbnail generator should only make 1 attempt to fetch them. Right now we have a post that is filled with identical broken image links (in this case to a spacer.gif) can suck up the build process for hours.

Add a 'page link format' option to the dashboard – the link format now only applies to entries. Tell Mike

Improvements to server infrastructure

• Work out way to recover from expired cert in the cache. To reproduce, consider a member who signed up, connected example.com, then moved the domain to another service. The attempt to renew the cert for example.com will fail. When the customer re-registers with Blot, for some reason open-resty-auto-ssl will serve an old certificate. I followed these steps to reset but we should automate this
• Create DMARC record for Blot https://en.wikipedia.org/wiki/DomainKeys_Identified_Mail https://docs.aws.amazon.com/ses/latest/DeveloperGuide/dmarc.html

Improvements to entry builder

• Improve the automatic title generated for screenshots with default MacOs filename from 'at 14:40pm'
• Generate a list of titles which you could use to generate a table of contents and tell Zach
• Use time draft was moved into blog folder for entry created/publish date, rather than the time the draft file itself was created. Tell Kerim and Shibel
• Parse Word Document subtitles as h2 and tell Matt
• Add way to filter entries by post type (e.g. photo) and tell Justin.
• If entries share publish date, sort them by blot creation date

Improvements to dashboard

• Add error messages on overdue subscription payment form
• Use <select> optgroup on timezone select dropdown to subdivide zones by current time, and steal the presentation format used by basecamp.
• Add a broken link checker
• Indicate that some posts' permalink will not change (if metadata) on the dashboard when setting the blog's permalink format. Tell Chet.
• Dumb quotes are still smartified, even without typeset enabled, possibly by pandoc?
• Undo option for form changes
• Add transactional emails notifying customer of changes to their email or password?
• When a customer pastes the whole tracking script into the preferences page on the dashboard, strip out everything except the ID which we need... perhaps create a page explaining what is happening before doing this.
• Handle noscript on dashboard / signup gracefully (remove all dashboard js?)
• Allow customers who are restarting an account with multiple blogs to disable some before restarting.

---

Add support for paths to static files with a trailing slash then tell Sean

Finish template translation offers

• Anders Norén
• Radek Kozieł
• Port Jane hugo theme to Blot and tell Jack and Josh on micro.blog
• Ghostwriter and tell Simon
• Work on Tim's template
• Translate this theme to Blot and tell Naveen
• Translate Ana's designs to Blot
• Translate 2019 to Blot and tell Gabriel
• Translate Paul's template into Blot
• Add a bootswatch template and tell Michael
• Port the jekyll template Neocactus to Blot and document the process then tell Robert

Add way to filter archives page by year or month and tell Chad

Adjust length of slug to use as few words as possible until collision and tell Chet

---

Improve 404 log page on Dashboard

• Make the 404 page more useful (see Netlify site)
• Add button to clear 404 log and tell Jamie
• Add frequency indicator for each route 404 log and tell Jamie
• Add filtering by date to 404 log and tell this person

---

Write new app/build/converters to turn new file formats into posts

• Add support for posts generated from Excel files, would generate a table. Use this library?
• Add AVIF image post support and tell Simon
• Add support for Ulysses .ulz file format and tell Nash and add support for .ulyz file created by Ulysses Adapt this script
• Add support for Jupyter Notebook .ipynb using Pandoc and tell Arnab and notebooks (via pandoc) and tell questioner
• Add support for .doc using libreoffice and tell Kyle
• Video posts (.mp4) - add way to opt-in to these
• Add support for .tiff images
• OPML files and tell John
• Audio posts (.mp3 for now?) and tell Kevin - add way to opt-in
• Add support for INI files and tell Frank

Write new clients to sync blog folders

• Add OneDrive client
  • Tell Nenad
  • Tell John
  • Tell Shantesh
  • Tell Nathan
  • Tell ekskog
  • Tell John
  • Tell Simon
  • Tell Andrew
  • Tell Naveen
  • Tell Emad
  • Tell John
  • If OneDrive for Business is supported tell Josh
• Add Apple Notes client
  • Tell Håvard
• rsync
  • Tell Harry
• SFTP
  • Tell Coraline
  • Tell Utkorg
  • Tell Calvin
  • Tell Fabrice
• NextCloud
  • Tell Vincent
  • Tell Coraline
  • Tell Andrew
  • Tell Mike
• Box
  • Tell Eric
• Syncthing
  • Tell Jason
• Github
  • Tell John
  • Make it possible to connect github repo via their API and tell reg
• Jumpshare
  • Tell Ömer
• pCloud
  • Tell Omaid
  • Tell Anders
  • Tell Tim
  • Tell JMN
  • Tell Rodney
• Dropmark
  • Tell Rado
• Proton Drive
  • Tell Ted
  • Tell Francois

---

Add password protection for blogs

• tell Ed, this person, this person, and Will, and Modomiro and Julian and Raph and Jeremy and Tara and Phil and Florian and Roy's friend Tim and Kaleb and Lotus and Khaled and Noel and Franz and Justin and Kyle and Devan and John and Anita and Emerald and Ben and Giuseppe and Annie and Jan
• think about how to add simple password (just a string) protection to sites.
• we need to consider both the node.js app (app/index) and the openresty reverse proxy (proxy/), especially the proxy caching behaviour for sites.
• perhaps we'll need to handle sessions for sites with password protection in some way at the open resty level? or just use some response header from upstream to always bypass the proxy cache? or some cookie? we don't want to add any extra work in openresty for sites without passwords. perhaps sites with passwords return responses with a header that prevents any proxy caching from taking place?
• anyway, it would involve:
• updating the blog model (password: string).
• adding a simple on/off input button to the dashboard, possibly under a new section.
• creating a password form view common to all sites, or able to be overwritten (password.html in a template perhaps).

Add server-side analytics:

• Tell Tom
• Tell John
• Tell Romain

Add commenting system

• Tell Precious
• Tell Hochan who is willing to beta test

Add email newsletter system

• Tell David
• Tell Nash
• Tell Kyle

Create an API for Blot

• Add webhooks and tell Aurynn and Shawn
• Use it to build a minimal blot cli sync client run from the command line (could background itself?!)

---

Self-hosting:

• Write self-hosting documentation
  • Tell Jake
  • Tell Berndl
  • Tell Joel
  • Tell Mohammed
  • Tell BP
  • Tell Rado
  • Tell Nuno
  • Tell Omaid
  • Tell Brian
  • Tell Jarod
  • Tell Francesco
  • Tell Pranay
  • Tell Aryan
  • Tell Todd
  • Tell Anita
  • Tell Kaushik
  • Tell Michael
  • Tell Frank
  • Tell Zip
• Tell Nick when it's possible to deploy a self-hosted Blot on Vercel and Zia