Files Bigger Than 16K, Again

Introduction

add refuses anything over 16K, because one directory entry can only describe 16K. S17 showed how the Einstein stores bigger files - one entry per 16K, numbered as extents - and taught edisc.py to read them. Writing them is the same idea backwards, and this section replaces S22's add_file with one that can do it.

Cutting A File Into Extents

A file of n records needs (n + 127) // 128 entries - rounding up, as in S22. Then, for extent e:

• its blocks are the file's blocks from number e x 8 to e x 8 + 7
• the next eight, or fewer for the last;
• its records are 128, except the last, which has whatever is left: records - e x 128. min(128, ...) gives the smaller of the two, so one line covers both cases;
• its extent number, byte 12, is e.

The blocks are still the lowest free ones, in order, so a big file usually sits in one continuous run of blocks - but nothing depends on that: each entry says exactly which blocks it holds.

Before writing anything, add_file now checks for enough free slots as well as enough blocks. A disc can run out of directory slots - 64 of them - while it still has space, if it holds many small files.

The code

Replace S22's add_file with this:

def add_file(disc, path, disc_name):
    base, ext = split_name(disc_name)
    if name_in_use(disc, base + "." + ext):
        sys.exit(f"{base}.{ext} is already on this disc")
    with open(path, "rb") as f:
        data = f.read()
    if len(data) == 0:
        sys.exit(f"{path} is empty: there is nothing to add")
    size = len(data)
    records = (size + 127) // 128
    data = data + bytes([0x1A]) * (records * 128 - size)
    count = (len(data) + 2047) // 2048
    extents = (len(data) + 16383) // 16384
    blocks = free_blocks(disc)[:count]
    slots = free_slots(disc)[:extents]
    if len(blocks) < count:
        sys.exit(f"The disc is full: {path} needs {count * 2}K")
    if len(slots) < extents:
        sys.exit(f"The disc's directory is full: {path} needs {extents} slot(s)")
    for i in range(count):
        chunk = data[i * 2048:i * 2048 + 2048]
        write_block(disc, blocks[i], chunk + bytes([0xE5]) * (2048 - len(chunk)))
    directory = bytearray(read_directory(disc))
    for extent in range(extents):
        part_blocks = blocks[extent * 8:extent * 8 + 8]
        part_records = min(128, records - extent * 128)
        slot = slots[extent]
        directory[slot * 32:slot * 32 + 32] = make_entry(base, ext, extent,
                                                         part_records, part_blocks)
    write_directory(disc, directory)
    print(f"Added {base}.{ext}: {size} bytes, {count} block(s), {extents} extent(s)")

The first half is S22's, without the 16K limit. The new part is the loop at the end, which writes one entry per extent. Nothing else in the program changes.

(The whole program: s23-edisc.py.)

Run it

STORY.TXT is the file you extracted from FILES.DSK in S18 - 20,096 bytes. Make a fresh disc with S22's two small files on it, then add the story:

python3 edisc.py create MINE3.DSK
python3 edisc.py add MINE3.DSK hello.py
python3 edisc.py add MINE3.DSK numbers.py NUMS.PY
python3 edisc.py add MINE3.DSK STORY.TXT
python3 edisc.py entries MINE3.DSK

What you should see

Created MINE3.DSK: a blank Einstein disc, 188K free
Added HELLO.PY: 89 bytes, 1 block(s), 1 extent(s)
Added NUMS.PY: 284 bytes, 1 block(s), 1 extent(s)
Added STORY.TXT: 20096 bytes, 10 block(s), 2 extent(s)
Slot  0: HELLO.PY      status 00  extent 0  records   1  blocks [1]
Slot  1: NUMS.PY       status 00  extent 0  records   3  blocks [2]
Slot  2: STORY.TXT     status 00  extent 0  records 128  blocks [3, 4, 5, 6, 7, 8, 9, 10]
Slot  3: STORY.TXT     status 00  extent 1  records  29  blocks [11, 12]

The same shape as STORY.TXT on FILES.DSK in S13 - 128 records and eight blocks, then 29 records and two - in different blocks, because MINE3.DSK already had two files. And the Einstein:

0:DIR 1:
1: HELLO   .PY  : NUMS    .PY
1: STORY   .TXT
24k Size, 164k Free, 190k Total

One STORY.TXT, its two entries counted together. To be sure the bytes are right too, take it off again and compare:

python3 edisc.py extract MINE3.DSK STORY.TXT back.txt

back.txt and STORY.TXT are identical, byte for byte. (Compare them with a short program, as in S18, or with cmp on macOS and Linux, fc /b on Windows.)

Change one thing

• Make a file of 40,000 bytes - a program that writes b"x" * 40000 to a file will do - and add it. How many extents, blocks and records? Work them out before you run entries.
• In the loop, change extent in make_entry(base, ext, extent, ...) to 0. Add a big file to a new disc and run type or extract on it. What does read_file say, and why?
• How many 1-byte files can you add to a blank disc before add refuses - and which check stops it, blocks or slots?

When it goes wrong

Summary

A file over 16K is written as one entry per 16K: eight blocks and 128 records to each, the last taking what is left, numbered 0, 1, 2 ... in byte 12. add now checks for enough slots and blocks before writing a byte, and the Einstein lists the result as one file.

Next

S24, remove - what deleting a file really changes, and what it does not.