generated from ztimson/template
188 lines
7.2 KiB
Markdown
188 lines
7.2 KiB
Markdown
<div id="top" align="center">
|
|
<br />
|
|
|
|
<!-- Logo -->
|
|
<img src="https://git.zakscode.com/repo-avatars/c44395fb50d10629bb79ae9294b43884679720fb589b69ea5fc854d790a6d8ec" alt="Logo" width="175" height="200">
|
|
|
|
<!-- Title -->
|
|
### Zim Utils
|
|
|
|
<!-- Description -->
|
|
Node.js wrapper around [kiwix-tools](https://github.com/kiwix/kiwix-tools) for managing a local ZIM library, serving content, and searching both locally and against the Kiwix catalog
|
|
|
|
<!-- Repo badges -->
|
|
[](https://git.zakscode.com/ztimson/zim-utils/tags)
|
|
[](https://git.zakscode.com/ztimson/zim-utils/pulls)
|
|
[](https://git.zakscode.com/ztimson/zim-utils/issues)
|
|
|
|
<!-- Links -->
|
|
|
|
---
|
|
<div>
|
|
<a href="https://git.zakscode.com/ztimson/zim-utils/wiki" target="_blank">Documentation</a>
|
|
• <a href="https://git.zakscode.com/ztimson/zim-utils/releases" target="_blank">Release Notes</a>
|
|
• <a href="https://git.zakscode.com/ztimson/zim-utils/issues/new?template=.github%2fissue_template%2fbug.md" target="_blank">Report a Bug</a>
|
|
• <a href="https://git.zakscode.com/ztimson/zim-utils/issues/new?template=.github%2fissue_template%2fenhancement.md" target="_blank">Request a Feature</a>
|
|
</div>
|
|
|
|
---
|
|
</div>
|
|
|
|
## Table of Contents
|
|
- [Zim Utils](#top)
|
|
- [About](#about)
|
|
- [Built With](#built-with)
|
|
- [Setup](#setup)
|
|
- [Production](#production)
|
|
- [Usage](#usage)
|
|
- [KiwixServer](#kiwixserver)
|
|
- [Catalog](#catalog)
|
|
- [License](#license)
|
|
|
|
## About
|
|
|
|
`@ztimson/zim-utils` manages a local [ZIM](https://wiki.openzim.org/wiki/ZIM_file_format) library by driving the official [kiwix-tools](https://github.com/kiwix/kiwix-tools) binaries (`kiwix-serve`, `kiwix-manage`) as child processes, rather than re-implementing ZIM reading from scratch. This means content serving, indexing, and fulltext search all come straight from Kiwix's own battle-tested implementation.
|
|
|
|
It comes with:
|
|
|
|
- **`KiwixServer`** — Owns a directory of `.zim` files: builds/rebuilds `library.xml`, starts/stops `kiwix-serve`, lists the local library, runs fulltext search, and resolves content URLs/bytes for any archive+page
|
|
- **Catalog** (`zimCatalog`/`zimCatalogInfo`) — Helper functions to search the remote Kiwix OPDS catalog, for discovering and downloading new ZIMs
|
|
|
|
### Built With
|
|
[](https://nodejs.org/)
|
|
[](https://javascript.com/)
|
|
[](https://github.com/kiwix/kiwix-tools)
|
|
|
|
## Setup
|
|
|
|
<details>
|
|
<summary>
|
|
<h3 id="production" style="display: inline">
|
|
Production
|
|
</h3>
|
|
</summary>
|
|
|
|
#### Prerequisites
|
|
- [Node.js](https://nodejs.org/en/download)
|
|
|
|
#### Instructions
|
|
1. Install the package: `npm install @ztimson/zim-utils`
|
|
2. On first install, `kiwix-serve`/`kiwix-manage` binaries are fetched into `./bin` (or wherever `binDir` points) - no separate Kiwix install needed.
|
|
|
|
</details>
|
|
|
|
## Usage
|
|
|
|
### KiwixServer
|
|
|
|
`KiwixServer` owns a directory of `.zim` files and manages the whole `kiwix-serve` lifecycle for you.
|
|
|
|
```js
|
|
import {KiwixServer} from '@ztimson/zim-utils';
|
|
|
|
const server = new KiwixServer('./zims'); // optional 2nd arg: {port, host, binDir}
|
|
await server.start(); // rebuilds library.xml, spawns kiwix-serve, waits until ready
|
|
await server.reload(); // pick up newly added/removed .zim files, no downtime
|
|
await server.stop(); // Kill server
|
|
```
|
|
|
|
#### List Local ZIMs
|
|
|
|
```js
|
|
const local = await server.list();
|
|
[
|
|
{
|
|
id: '37a99758-43a0-6ba3-cd54-1af556369eee',
|
|
title: 'FOSS cooking',
|
|
updated: 2026-05-06T00:00:00.000Z,
|
|
summary: 'Making cooking fast, easy, foss',
|
|
language: 'eng',
|
|
name: 'foss.cooking_en_all',
|
|
category: 'other',
|
|
tags: ['_category:other', '_ftindex:yes', 'preppers,food', '_pictures:yes', '_videos:yes', '_details:yes'],
|
|
mediaCount: 154,
|
|
author: '-',
|
|
publisher: 'openZIM',
|
|
articleCount: 719,
|
|
sizeMb: 23.1,
|
|
href: 'foss.cooking_en_all_2026-05',
|
|
icon: undefined,
|
|
viewer: 'http://127.0.0.1:51992/content/foss.cooking_en_all_2026-05',
|
|
},
|
|
// ...
|
|
]
|
|
```
|
|
|
|
#### Search Local ZIMs
|
|
|
|
```js
|
|
// Fulltext search across every local archive: xapian prefilter, then fuzzy re-ranked
|
|
const hits = await server.search('chocolate', 5);
|
|
[
|
|
{
|
|
id: '37a99758-43a0-6ba3-cd54-1af556369eee',
|
|
title: 'Chocolate Chip Cookies',
|
|
page: 'foss.cooking/recipe/mfed3/chocolate-chip-cookies',
|
|
name: 'foss.cooking_en_all',
|
|
publisher: 'openZIM',
|
|
href: 'foss.cooking_en_all_2026-05/foss.cooking/recipe/mfed3/chocolate-chip-cookies',
|
|
icon: undefined,
|
|
viewer: 'http://127.0.0.1:51992/content/foss.cooking_en_all_2026-05/foss.cooking/recipe/mfed3/chocolate-chip-cookies',
|
|
summary: '...cream scooper or spoon to make uniform balls of cookie dough...',
|
|
score: 0.67,
|
|
},
|
|
// ...
|
|
]
|
|
```
|
|
|
|
#### View ZIM Content
|
|
|
|
```js
|
|
// Build a content URL from any href (list()/search() output, or a full viewer URL) without hitting the network
|
|
const url = server.fetch(hits[0].href);
|
|
'http://127.0.0.1:51992/content/foss.cooking_en_all_2026-05/foss.cooking/recipe/mfed3/chocolate-chip-cookies'
|
|
|
|
// Or fetch the bytes directly, proxied straight from kiwix-serve
|
|
const {mimetype, data} = await server.raw(hits[0].href);
|
|
{mimetype: 'text/html; charset=utf-8', data: <Buffer ...>}
|
|
```
|
|
|
|
### Catalog
|
|
|
|
```js
|
|
import {zimCatalog, zimCatalogInfo, CATALOG_URL} from '@ztimson/zim-utils';
|
|
|
|
// Search the remote Kiwix catalog to discover new ZIMs
|
|
const search = await zimCatalog('knots', {lang: 'eng', count: 20, url: CATALOG_URL});
|
|
// OR Exact lookup by catalog `name`, useful for checking if a local copy is outdated
|
|
const entry = await zimCatalogInfo('wikipedia_en_knots');
|
|
|
|
[
|
|
{
|
|
id: 'urn:uuid:3a4fe0d0-0bd0-7583-ada8-c52d173ae44d',
|
|
title: 'Knots by Wikipedia',
|
|
updated: 2026-07-20T00:00:00.000Z,
|
|
summary: 'A subset of Wikipedia encyclopedia dedicated to knots',
|
|
language: 'eng',
|
|
name: 'wikipedia_en_knots',
|
|
category: 'wikipedia',
|
|
tags: ['wikipedia', '_category:wikipedia', '_pictures:yes', '_videos:no', '_details:yes', '_ftindex:yes'],
|
|
mediaCount: 3501,
|
|
author: 'Wikipedia',
|
|
publisher: 'openZIM',
|
|
articleCount: 1730,
|
|
sizeMb: 17.6,
|
|
href: 'https://lb.download.kiwix.org/zim/wikipedia/wikipedia_en_knots_maxi_2026-07.zim.meta4',
|
|
icon: 'https://library.kiwix.org/catalog/v2/illustration/3a4fe0d0-0bd0-7583-ada8-c52d173ae44d/?size=48',
|
|
viewer: 'https://library.kiwix.org/viewer#wikipedia_en_knots',
|
|
},
|
|
// ...
|
|
]
|
|
|
|
// Download catalog ZIM if out of date (unless forced) and reload automatically
|
|
await mgr.download(search[0].href, force);
|
|
```
|
|
|
|
## License
|
|
Copyright © 2026 Zakary Timson | Available under MIT Licensing
|