Chapter 3. From Input to Vault — A Four-Stage Flow

Why can’t we process everything at once? Why not spread out the IP documents and create the final deliverable immediately? In Obsidian, that final deliverable is called a Vault. Since this is the first appearance of the term, its exact form will be introduced shortly in “The Landscape of Output.” The reason Librarying has four stages is the answer to that question.

Librarying begins with input, passes through first extraction, second integration, third skeleton building, and fourth narrative writing, and ends in a Vault. We will look in turn at what each stage receives and creates, what decisions intervene between them, and where and how people and AI work partners divide the work.

The Landscape of Input

Picture the desk where the work begins. Rulebooks from editions one through three are piled up; there is a folder of short stories from five years; there is an external wiki someone began and then abandoned; and there is a naming candidate that appeared once in a collaboration chat and vanished. The input Librarying receives is all of this scattered material. Every document belonging to an IP enters as input.

Rather than follow one particular IP to the end, this chapter will briefly quote the example best suited to each stage. To ensure that the characters, devices, and events in these examples do not directly overlap with any reader’s IP, they will be presented as an anonymous “certain IP.” If, while reading, that desk begins to overlap with your own, that overlap is the passage through which this procedure enters your IP.

If there are rulebooks, every edition—first, second, and third—is input, including a discarded zeroth edition. A noun or point of view born in discarded material may have flowed into a surviving short story, so discarded editions must pass through once in order to catch those traces. Short stories, novels, and scenarios are all works the IP has operated, and for works in other languages, the original text goes in as well. Planning documents and design documents are also input: a planning document organizes a work’s broad picture and intentions, while a design document contains decisions not directly exposed in the work, such as a character’s trauma, an event’s causal chain, or a device’s operating principle.

External wikis pass through as well. These are pages someone began five years ago and then stopped updating; both maintained and neglected pages are included in the input. Neglected pages preserve nouns and points of view from five years ago, and they must be examined once to determine whether those terms conflict with current works.

If the work is a video or a game, scene-by-scene dialogue data, cutscene scripts, and in-game text are input. So are paper notes tucked beside the desk, text in temporary folders, and a naming candidate that appeared once in a collaboration-chat log. Even material that is not official passes through for now, because the acceptance threshold for first extraction is generous.

The volume of input differs by IP. For a mid-scale IP operated for five years, the combined body text runs from one to five million Korean characters. At an average of roughly 300,000 characters per Korean trade paperback, that means input equivalent to three to seventeen books, and one person would need a month merely to read it all once. Librarying draws keywords from this input, but people do not perform that complete reading; if they tried to extract through full reading, the work would expand back into a six-month project.

The first task in organizing input is building a location catalog. In a single table, record which documents are in which directories, what period of the IP each document belongs to, and what domain it belongs to—rules, short fiction, planning, or external material. This catalog is the starting point of first extraction, and people build it. You can hand the directory tree to an AI work partner and receive a catalog, but a person decides which domain each document belongs to. If that judgment blurs, extraction in the next stage blurs as well.

The Landscape of Output

Librarying outputs an Obsidian Vault. In Obsidian, a Vault is a collection of .md files: .md files live in one working folder and are connected to one another through wikilinks. If you are new to Obsidian, think of a Vault as one whole library for an IP—a building with 1,500 books on its shelves, each connected to others by citations.

When you first open an IP’s Vault, the left sidebar lists 1,000 to 2,000 .md files alphabetically or in Korean alphabetical order. Above it is a search box, and beside it a button for graph view. Selecting a file opens its body, which has five parts: frontmatter (YAML metadata) at the head; a one- or two-line definition; two or three paragraphs of detailed explanation; multilingual translation candidates; and a 200- to 1,000-character narrative written by an AI work partner.

Frontmatter is a set of lines pairing keys and values with colons. Representative keyword, category, worldview axis, English name, romanization, version status, source, and related keywords appear one per line. The worldview axis is the name of this book’s second classification criterion and receives a full chapter in Chapter 6. People rarely read frontmatter directly; search and automation read it to do their work. Definitions take one or two lines; detail takes one paragraph when short and two or three when longer; translation candidates provide multilingual forms such as English and romanization; and the narrative block contains 200 to 1,000 characters according to an entry’s importance. That is the shape of one .md file, and all 1,500 files are organized in that form.

Pressing Obsidian’s graph-view button spreads 1,500 points and the lines between them across one screen. A point corresponds to one headword, and a line to one wikilink. A point with many lines converging on it is a hub; one with few is a leaf. This constellation-like image is Librarying’s final deliverable.

Once completed, the Vault becomes the IP’s source of truth. Deep-reading work on foreign-language works cites the Vault’s character and device entries; work that creates characters from zero refers to its worldview keywords; and publication LOREBOOK production turns its 1,500 headwords into encyclopedia entries. Chapter 12 examines this output landscape in detail.

The flow is not linear. Vocabulary newly created in output work, newly confirmed spellings, and newly added settings flow back into the Vault; this is called backflow. Whenever backflow arrives, a mini-cycle of first extraction and second integration repeats, and third skeleton building is updated incrementally. The Vault is therefore not finished once and for all: it keeps growing while the IP lives. Chapter 11 examines backflow and the operating cycle in detail.

Why Must It Be This Order?

Before introducing the four stages, I want to answer one question: why four stages? Couldn’t they be reduced to two or three?

Once you understand why each stage occupies its place in the order, you can see how to adjust each stage when applying Librarying to your own IP.

Each stage solves a problem at a different level. The first asks, “What exists?” and pulls keywords themselves from the input documents. The second moves on to “Are like things grouped together?”, merging equivalent forms under one headword and selecting a representative notation. At the third, the question changes to “Is it in a form tools can handle?”, converting CSV rows into Obsidian .md files. The final fourth stage asks, “Is this living information?” and fills in narratives explaining the place a headword occupies within the IP.

Because the levels differ, the order cannot be reversed. If you begin with second integration, there is no material to group and the work cannot start. If you skip the second stage and send the first stage’s 5,000 rows directly to the third, the same object is scattered across three .md files, none of which cites the others. If you skip the third stage, 2,000 headwords sit in CSV rows but there are no .md files with space for fourth-stage narratives. If you skip the fourth, there is frontmatter, definition, and detail but no narrative that explains an entry’s place in the IP; output work then receives a dictionary with information but no context.

Data engineering has a framework called ETL: Extract, Transform, Load. It is an asset refined over decades for extracting, transforming, and loading data. Librarying’s first extraction corresponds to Extract, second integration to Transform, and third skeleton building precisely to Load.

It is tempting to say that the first three stages of Librarying are merely an application of ETL. I see it the other way around. Data engineering has used this framework for structured and semi-structured materials such as transaction records, logs, and sensor readings. IP narrative material, however, is material no one has placed into this framework before. The first, second, and third stages are what sharpen the mixed materials of rulebooks, short fiction, discarded editions, and external wikis—materials full of human touch, contradiction, and changes over time—into the extract-transform-load frame. They are not something merely similar, but the result of borrowing a proven framework for a new material and making it specific to that use.

And ETL has no fourth stage. Where data engineering ends with loading, Librarying adds one further layer of living narrative on the organized material. This fourth stage, which does not stop at organization but writes upon it, is Librarying’s distinctiveness; no data pipeline has an equivalent stage.

What the Four Stages Make

Let us briefly identify the input each stage receives and the output it creates. Chapters 4 through 9 unfold one stage at a time in detail.

First extraction pulls every keyword from input documents and organizes them into a five-column CSV. CSV is a tabular text format that separates fields with commas; because both spreadsheets and automation tools can handle it, it suits an intermediate deliverable. Here it has five columns—IDX, category, keyword, description, and source—and one IP produces 5,000 to 15,000 rows. The acceptance threshold is generous: duplicates, typos, and lack of refinement are all allowed. Trying to extract accurately from the start forces you, for every keyword, to ask at once whether it duplicates an existing headword, which category it belongs to, and what representative notation should contain it. That bottleneck, in which one decision waits for the next, makes first extraction dozens of times slower. Like RAW photography, capture everything first; selection and correction come in the next stage. AI work partners handle the core task, automatically extracting nine category tables in parallel. People refine the nine-category guide, decide new categories, review at a macro level, and check traces of pronouns, points of view, and discarded nouns. Chapter 4 covers this in detail.

Second integration groups the 5,000 to 15,000 rows from the first stage into 1,000 to 2,000 headwords and forms a thirteen-column CSV. Its acceptance threshold is demanding—95 percent—because a wrong decision throws the entire wiki out of alignment. The most common decision is a synonym cluster: grouping “black void,” “black smoke,” and “dark fog” under one headword and selecting a representative notation. Judgments about homonyms, keywords whose meanings change over time, and different characters with the same name can be made accurately only in the mind of someone who has read the five years of work. An AI work partner quickly proposes synonym candidates, definition drafts, related links, and worldview-axis candidates; the person makes the final decision. The thirteen-column CSV is organized in roughly two to four hours. Chapters 5 and 6 cover this in detail.

Third skeleton building converts the second-stage CSV into 1,500 Obsidian .md files. It is almost fully automated, and because it is a deterministic conversion, little human touch is needed.

One headword in one row becomes one .md file. The row’s columns become frontmatter fields; definition and detail columns become body paragraphs; related keywords are automatically injected as wikilinks; and an empty space remains at the end for narrative. At this point, opening Obsidian graph view displays a network of 1,500 nodes connected by wikilinks on one screen. The writing space, however, is still empty.

The build itself takes about five minutes, followed by about thirty minutes of human review. The review target is a build-result summary called a manifest, containing the number of files, category counts, and progress status (covered in Chapter 7). If a conversion rule has a flaw, all 1,500 files inherit the same flaw, so human inspection after the build is mandatory. Chapters 7 and 8 cover this in detail.

Fourth narrative writing is Librarying’s creative stage. Where the first, second, and third stages erect a skeleton, the fourth writes the narrative that makes the IP world live upon it. Each headword receives 200 to 1,000 characters: hubs receive 500 to 1,000 and leaves 200 to 300. AI work partners draft 1,500 narratives in roughly five to ten hours (more than 1,000 hours by hand), while people review form, links, length, tone, and project context and raise them to the acceptance threshold. A leaf may be short, but that does not mean it may be empty; an operating Vault targets at least 95 percent overall LLM completion. Chapter 9 covers this in detail.

The Nature of Stages One, Two, and Three; the Nature of Stage Four

After laying out these four stages as one flow, one point deserves emphasis: stages one, two, and three differ in nature from stage four.

The essence of the first three stages is mechanical organization. They pull keywords from input, group equivalent meanings, and convert them into Obsidian form. Clear rules exist, and the result is determined when those rules are followed. People make decisions too, but their range is narrow and clear.

The essence of the fourth stage is narrative writing: writing an account that unfolds one headword as a flow. It cannot be reduced to clear rules, and the acceptance threshold for one headword is not the same as the threshold for the next. Tone, context, and the IP’s own subtleties all intervene.

This difference in nature determines the texture of collaboration. In stages one, two, and three, we hand input to AI work partners, receive the results, and pass them almost directly to the next stage. In stage four, we receive a first draft and make two or three rounds of exchange.

Time Budget

Adding the time for stages one through four, Librarying for one IP takes roughly nine to sixteen hours. This number is the actual nature of the time reduction this book shows the reader.

The allocation is as follows. Profiling—worldview axes, category guides, and a reference-file catalog—takes thirty to sixty minutes. First extraction takes thirty minutes to two hours (parallel AI work); second integration takes two to four hours (human decisions with AI assistance); third skeleton building takes five to ten minutes (deterministic automation); story-digest preparation takes thirty to sixty minutes. This digest is not a simple summary meant to avoid rereading the whole manuscript; it is a file that interprets manuscripts, scripts, and rulebooks from the producer’s viewpoint and compresses them into context for fourth-stage prose. Fourth narrative writing takes five to ten hours (AI work partners at the core), and verification takes one to three hours (human review with automated checks). The total is nine to sixteen hours, measured for the first cycle.

The first cycle is the time for Librarying an IP for the first time. Later, when a new short story arrives for an incremental update, the time is much shorter. A short story usually brings thirty to fifty new headwords; a mini-cycle of stages one, two, and three takes about thirty minutes, and adding fourth-stage narratives takes about an hour. Thus an update for one short story finishes in a total of 1.5 to 2 hours. Chapter 11 examines this operating cycle in detail.

A Diagram of How Two Hands Divide the Work

Putting this diagram into one flow makes it easy to see where each chapter from 4 through 9 looks.

Profiling is human work: deciding worldview axes, refining the category guide, and building the reference-file catalog. In first extraction, AI work partners automatically extract nine category tables while people handle macro review and new-category decisions; the acceptance threshold is 60 percent. In second integration, people handle synonym clusters, headword naming, and worldview-axis confirmation, while AI produces synonym candidates, definition drafts, and worldview-axis candidates; the acceptance threshold is 95 percent. In third skeleton building, a script runs and people check the manifest and automated-verification results; the acceptance threshold is 100 percent. In fourth narrative writing, AI drafts each entry and people provide the tone guide, five-stage review, and acceptance decisions. Hubs are written long and deep; leaves short and precise. In verification, automated checks write violation reports while people conduct manual sampling and judge the integrated build; the acceptance threshold is 100 percent.

This diagram guides the fourteen chapters. Chapters 4 through 9 look closely at each row above; Chapter 10 covers verification, Chapter 11 the operating cycle, Chapter 12 the outputs, Chapter 13 a reflection on five years of operation, and Chapter 14 the conclusion.

Let us briefly clarify hub and leaf. Not all 1,500 headwords carry equal weight. A headword at the core of an IP is often cited by other headwords. Such a frequently cited headword is called a hub; one with few citations is called a leaf. Of 1,500 headwords, usually 100 to 150 are hubs. Narrative length and review thresholds apply differently: hubs are written long and deep, leaves short and simple. Chapter 8 examines automatic hub classification and human decisions in detail.

Holding the Diagram in Hand

The first stage gathers materials, the second organizes them, the third converts the organized materials into a form tools can handle, and the fourth fills that form with the IP’s narratives. Reverse this order and the work collapses; follow it and the result is nine to sixteen hours.

Beginning with the next chapter, Chapter 4, we enter first extraction. Across a whole chapter, we will look closely at what the nine categories separate and what they do not combine, how one row of a five-column CSV is formed, and what happens when an AI work partner performs exhaustive extraction on normalized input documents.