package_my_video: debug faststart

posts/package_my_video.md

@@ -274,5 +274,182 @@ Now regenerate the frame dump and check if our I frames match the expected: 1, 7
 
 Excellent!
 
+Let's check where the mp4 "atoms" are located in the resulting file.
+
+```
+❯ ffprobe -v trace ./test-videos/bbb_h264_aac.mp4 2>&1 | grep 'type:.\(ftyp\|free\|mdat\|moov\)'
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x90d7a80] type:'ftyp' parent:'root' sz: 32 8 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x90d7a80] type:'free' parent:'root' sz: 8 40 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x90d7a80] type:'mdat' parent:'root' sz: 157264899 48 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x90d7a80] type:'moov' parent:'root' sz: 412246 157264947 157677185
+```
+
+So the `moov` atom is at the end of the file by default.
+
+Save this version of the transcode if you want to test how this works in the browser.
+
+To optimize for faster startup, there is a `faststart` option available which moves the `moov` atom to the head of the file.
+
+So adjusting the progressive script
+
+```diff
+diff --git a/progressive.py b/progressive.py
+index 0ba58b7..a3dc63a 100755
+--- a/progressive.py
++++ b/progressive.py
+@@ -36,6 +36,8 @@ def run_ffmpeg_transcode(infname, outfname, probeinfo, segment_length=3):
+                'libx264',
+                '-x264opts',
+                f'keyint={keyint}:min-keyint={keyint}:no-scenecut',
++               '-movflags',
++               'faststart',
+                '-acodec',
+                'aac',
+```
+
+And after the re-transcode:
+
+```
+❯ ffprobe -v trace ./test-videos/bbb_h264_aac.mp4 2>&1 | grep 'type:.\(ftyp\|free\|mdat\|moov\)'
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x34423a80] type:'ftyp' parent:'root' sz: 32 8 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x34423a80] type:'moov' parent:'root' sz: 412246 40 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x34423a80] type:'free' parent:'root' sz: 8 412286 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x34423a80] type:'mdat' parent:'root' sz: 157264899 412294 157677185
+```
+
+It worked!
+
+Lets prove out why this is great for browser playback.
+
+### faststart testing
+
+[`caddy`][caddy_files] has a nice quick built in file server with verbose access logs.
+
+Drop [this `index.html`][test_index] into the same directory as your test videos.
+
+```
+❯ caddy file-server --access-log --browse --listen :2015 --root ./test-videos
+```
+
+I kept my version of the mp4 prior to adding the `faststart` option, so I have two files:
+
+```
+❯ ffprobe -v trace ./test-videos/bbb_h264_aac.mp4 2>&1 | grep 'type:.\(ftyp\|free\|mdat\|moov\)'
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x15b64a80] type:'ftyp' parent:'root' sz: 32 8 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x15b64a80] type:'moov' parent:'root' sz: 412246 40 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x15b64a80] type:'free' parent:'root' sz: 8 412286 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x15b64a80] type:'mdat' parent:'root' sz: 157264899 412294 157677185
+
+❯ ffprobe -v trace ./test-videos/bbb_h264_aac_endmov.mp4 2>&1 | grep 'type:.\(ftyp\|free\|mdat\|moov\)'
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x89b2a80] type:'ftyp' parent:'root' sz: 32 8 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x89b2a80] type:'free' parent:'root' sz: 8 40 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x89b2a80] type:'mdat' parent:'root' sz: 157264899 48 157677185
+[mov,mp4,m4a,3gp,3g2,mj2 @ 0x89b2a80] type:'moov' parent:'root' sz: 412246 157264947 157677185
+```
+
+Now plugging in <http://localhost:2015/bbb_h264_aac_endmov.mp4> to the form:
+
+In firefiox there are 3 requests made:
+
+```
+# 1 req
+GET /bbb_h264_aac_endmov.mp4 HTTP/1.1
+Host: localhost:2015
+Accept: video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
+Range: bytes=0-
+# 1 resp
+HTTP/1.1 206 Partial Content
+Accept-Ranges: bytes
+Content-Length: 157677185
+Content-Range: bytes 0-157677184/157677185
+Content-Type: video/mp4
+Etag: "sfecu12lvkht"
+Content-Type: video/mp4
+```
+
+Note the amt transfered in first request is actually only 1.57 MB as reported in devtools.
+
+```
+# 2 req
+GET /bbb_h264_aac_endmov.mp4 HTTP/1.1
+Host: localhost:2015
+Accept: video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
+Range: bytes=157253632-
+# 2 resp
+HTTP/1.1 206 Partial Content
+Accept-Ranges: bytes
+Content-Length: 423553
+Content-Range: bytes 157253632-157677184/157677185
+Content-Type: video/mp4
+```
+
+157677184 is the last byte -1, so it is reading the last 423.83 kB of the file.
+
+```
+# 3 req
+GET /bbb_h264_aac_endmov.mp4 HTTP/1.1
+Host: localhost:2015
+User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:127.0) Gecko/20100101 Firefox/127.0
+Accept: video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
+Accept-Language: en-US,en;q=0.5
+Range: bytes=131072-
+# 3 resp
+HTTP/1.1 206 Partial Content
+Accept-Ranges: bytes
+Content-Length: 157546113
+Content-Range: bytes 131072-157677184/157677185
+Content-Type: video/mp4
+```
+
+Lastly, start reading at byte 131072 to the end of the file.
+
+A rough guess about how this works.
+
+Take a look at annotated byte sizes to the `ffprobe -v trace` from above as they match up with the range requests:
+
+```
+# the format of the numbers is: {size} {start_byte} {total_size}
+# 1 req     type:'ftyp' parent:'root' sz: 32 8 157677185
+# 1 req     type:'free' parent:'root' sz: 8 40 157677185
+# 1+3 req   ype:'mdat' parent:'root' sz: 157264899 48 157677185
+# 2 req     type:'moov' parent:'root' sz: 412246 157264947 157677185
+```
+
+* `# 1 req` fetches the first 1.57MB in a 206 partial content read from the head of the file.
+    * Looking for a `moov` atom for file information so it can start playing.
+    * This example video `moov` is 412 kB, so it's reading about 3x that and into the `mdat` section where the video data lives.
+* `# 2 req` fetches the last 423.83 kB from the end of the file.
+    * It hits the `moov`
+* `# 3 req` fetches whole file starting at 131.072 kB from the beginning of file.
+
+Pretty cool, you can see it hunting for the `moov` then starting playback.
+
+In contrast, here's the `faststart` option: <http://localhost:2015/bbb_h264_aac.mp4>
+
+```
+# 1 req
+GET /bbb_h264_aac.mp4 HTTP/1.1
+Host: localhost:2015
+Accept: video/webm,video/ogg,video/*;q=0.9,application/ogg;q=0.7,audio/*;q=0.6,*/*;q=0.5
+Accept-Language: en-US,en;q=0.5
+Range: bytes=0-
+# 1 resp
+HTTP/1.1 206 Partial Content
+Accept-Ranges: bytes
+Content-Length: 157677185
+Content-Range: bytes 0-157677184/157677185
+Content-Type: video/mp4
+```
+
+Same exact start to the flow - just read whole file with `Range: bytes=0-`.
+
+But this time firefox transfers ~7-9 MB (it changes per test), and there's only 1 request.
+
+Best guess here is that firefox is still trying to read 1.5MB, but it encounters the `moov` immediately and just keeps reading.
+
+That's the first time I've seen this in action.
+
 [pic_types]: https://en.wikipedia.org/wiki/Video_compression_picture_types
 [apple_hls_seg]: https://developer.apple.com/documentation/http-live-streaming/hls-authoring-specification-for-apple-devices#Media-Segmentation
+[caddy_files]: https://caddyserver.com/docs/quick-starts/static-files#command-line
+[test_index]: https://git.sr.ht/~cfebs/vidpkg/tree/main/item/test-videos/index.html