[{"data":1,"prerenderedAt":173},["ShallowReactive",2],{"article-hash-web-crypto-api-large-file":3},{"article":4,"relatedArticles":125},{"slug":5,"category":6,"date":7,"readingMinutes":8,"tags":9,"relatedTool":14,"cn":15,"en":64,"author":113,"reviewedBy":116,"reviewedDate":118,"updatedDate":118,"sources":119,"noIndex":124},"hash-web-crypto-api-large-file","实现原理","2026-07-28",5,[10,11,12,13],"Web Crypto API","Large File Hash","Async","Performance","/tools/dev/hash",{"title":16,"description":17,"intro":18,"sections":19},"Web Crypto 计算文件 SHA-256：非流式 digest 的内存边界与大文件方案","澄清 Web Crypto API 不支持流式 digest：中小文件可设置上限后一次性计算，真正的大文件应在 Worker 中使用可信的增量 SHA-256 实现。","在断点续传或去重判定中，前端常要计算文件指纹。浏览器原生 `crypto.subtle.digest` 支持 SHA-256，但它没有 `.update()`，调用前必须把完整输入放进内存。因此 `Blob.slice()` 本身不能让原生 digest 变成流式哈希。本文分别给出有明确大小上限的原生方案，以及真正大文件需要采用的增量 Worker 架构。",[20,24,29,33,37,42,46,50,54,60],{"heading":21,"paragraphs":22},"一、问题概述：大文件哈希计算的峰值内存与交互风险",[23],"对多 GB 文件调用 `file.arrayBuffer()` 会额外分配与文件大小相当的连续缓冲区，可能触发分配失败、进程被系统终止或严重内存压力。即便 `subtle.digest()` 返回 Promise，也不代表大输入没有 CPU 与资源竞争；文件大小策略和目标设备实测仍不可省略。",{"heading":25,"paragraphs":26,"code":28},"二、最小复现：盲目全量读取大文件与 Web Crypto API 的限制",[27],"下面的代码说明了一次性读取大文件以及混淆 `SubtleCrypto.digest` 支持算法的常见失误：","/* 错误 1：试图用 SubtleCrypto 计算 MD5 -> 直接报错 DOMException (Algorithm Not Supported) */\n// await window.crypto.subtle.digest(\"MD5\", data); // SubtleCrypto 根本不支持 MD5！\n\n/* 错误 2：一次性读取 2GB 文件传入 ArrayBuffer -> 页面内存爆满 OOM 崩溃！ */\n// const buffer = await file.arrayBuffer(); // 不可取！",{"heading":30,"paragraphs":31},"三、根因分析：SubtleCrypto 的算法与输入模型限制",[32],"1. **算法限制**：`subtle.digest()` 支持 `SHA-1`, `SHA-256`, `SHA-384`, `SHA-512`，不支持 MD5。2. **没有增量接口**：原生 API 单次接收完整的 BufferSource，不能逐块更新同一个摘要状态。3. **内存边界**：完整文件缓冲区、浏览器内部处理和页面其他对象会共同抬高峰值内存；具体上限不是 Web Crypto 规范的一部分，不能写死为某个浏览器固定值。",{"heading":34,"paragraphs":35},"四、推荐方案：先按文件大小选择正确的哈希实现",[36],"1. 中小文件：根据目标设备制定可配置的大小上限，在上限内使用 `file.arrayBuffer()` 与 `crypto.subtle.digest('SHA-256', buffer)`。2. 真正的大文件：在 Web Worker 中使用经过审计、支持增量 `.update(chunk)` 的 SHA-256 JS/Wasm 库，逐块读取并顺序更新同一个哈希状态；不能独立哈希每个切片后再拼接摘要。3. 进度与取消：主线程只接收进度和最终摘要，并支持终止 Worker。Worker 避免 UI 阻塞，但不会自动消除哈希状态和切片所需内存。",{"heading":38,"paragraphs":39,"code":41},"五、完整代码：带文件上限的原生 Web Crypto SHA-256",[40],"下面的代码忠实反映原生 API 的边界：它只接受上限内的文件，并一次性读取完整内容。`64 MiB` 是示例产品策略，不是浏览器保证；上线时应按支持的设备实测调整。超出上限后应切换到增量 Worker 实现，而不是假装 `Blob.slice()` 能流式调用 `digest()`。","const DEFAULT_NATIVE_DIGEST_LIMIT = 64 * 1024 * 1024;\n\nasync function computeFileSha256WithWebCrypto(\n  file: File,\n  maxBytes = DEFAULT_NATIVE_DIGEST_LIMIT,\n): Promise\u003Cstring> {\n  if (file.size > maxBytes) {\n    throw new RangeError(\n      \"文件超出原生 digest 的产品内存上限，请改用 Worker 中的增量 SHA-256 实现\",\n    );\n  }\n\n  const bytes = await file.arrayBuffer();\n  const digest = await crypto.subtle.digest(\"SHA-256\", bytes);\n  return Array.from(new Uint8Array(digest), (byte) =>\n    byte.toString(16).padStart(2, \"0\"),\n  ).join(\"\");\n}",{"heading":43,"paragraphs":44},"六、常见错误方案",[45],"使用 `SubtleCrypto.digest('MD5', buffer)` 期望计算 MD5 导致运行时捕获 DOMException 异常；在 UI 主线程循环中无休止地阻塞同步计算哈希导致页面失去响应；不检查文件物理大小盲目全量加载。",{"heading":47,"paragraphs":48},"七、边界条件：设备差异、Worker 与内存所有权",[49],"不要依赖未经规范保证的固定浏览器内存阈值；移动设备、并发标签页和浏览器版本都会影响可用内存。向 Worker 发送切片时可转移 ArrayBuffer 所有权以减少复制，但必须确认增量库不会在内部长期保留所有切片。页面关闭、用户取消和 Worker 异常也要能清理资源。",{"heading":51,"paragraphs":52},"八、如何验证文件哈希正确性与性能",[53],"使用系统 `shasum -a 256` 或服务端 SHA-256 与浏览器结果比对；固定测试空文件、非整块大小、含零字节文件和超过上限文件。对增量 Worker 方案监控峰值内存、主线程长任务、取消行为和不同大小下的进度单调性。",{"heading":55,"paragraphs":56},"九、FAQ",[57,58,59],"问：`Blob.slice()` 后逐块调用 `subtle.digest()` 再拼起来可以吗？答：不可以，那得到的是多个独立摘要，不等于整个文件的 SHA-256。","问：如何处理 2GB 文件？答：选择支持增量更新的可信 SHA-256 JS/Wasm 实现，在 Worker 中顺序读取切片并更新同一个状态；同时核对库的供应链、许可和测试向量。","问：Worker 能防止 OOM 吗？答：不能保证。它主要隔离计算，仍需限制切片大小、避免保留全部切片，并实测峰值内存。",{"heading":61,"paragraphs":62},"十、总结",[63],"原生 Web Crypto `digest()` 是非流式接口。中小文件应设置明确上限后一次性计算；超大文件应采用 Worker 加可信增量哈希库，并用标准测试向量与资源监控验证，不能用空循环切片伪装成流式 SHA-256。",{"title":65,"description":66,"intro":67,"sections":68},"File SHA-256 with Web Crypto: Memory Bounds and a Real Large-File Strategy","Explain that Web Crypto digest is non-streaming: use it only below an explicit file limit, and use a vetted incremental SHA-256 implementation in a Worker for genuinely large files.","Front-end upload and deduplication flows often need a file fingerprint. Native `crypto.subtle.digest` supports SHA-256 but has no `.update()` method: the complete input must be in memory before the call. `Blob.slice()` alone cannot turn it into streaming hashing. This guide separates a size-bounded native path from the incremental Worker architecture required for genuinely large files.",[69,73,78,82,86,91,95,99,103,109],{"heading":70,"paragraphs":71},"1. Problem: peak memory and interaction risks during file hashing",[72],"Calling `file.arrayBuffer()` on a multi-gigabyte file allocates another contiguous buffer roughly as large as the file and may fail, trigger process termination, or create severe memory pressure. The fact that `subtle.digest()` returns a Promise does not eliminate CPU and resource contention, so explicit size policy and device testing are still required.",{"heading":74,"paragraphs":75,"code":77},"2. Minimal reproduction: indiscriminate file loading and Web Crypto limits",[76],"This snippet illustrates common pitfalls when attempting full file reads and confusing `SubtleCrypto.digest` algorithm support:","/* Defect 1: attempting MD5 with SubtleCrypto -> throws DOMException (Algorithm Not Supported) */\n// await window.crypto.subtle.digest(\"MD5\", data); // SubtleCrypto does NOT support MD5!\n\n/* Defect 2: loading a 2GB file entirely into ArrayBuffer -> OOM browser crash! */\n// const buffer = await file.arrayBuffer(); // Dangerous!",{"heading":79,"paragraphs":80},"3. Root cause: SubtleCrypto algorithm and input-model limits",[81],"1. **Algorithm limits**: `subtle.digest()` supports `SHA-1`, `SHA-256`, `SHA-384`, and `SHA-512`; MD5 is omitted. 2. **No incremental API**: one call accepts a complete BufferSource and cannot update one digest state chunk by chunk. 3. **Memory boundary**: the file buffer, browser-internal processing, and the rest of the page contribute to peak memory. No fixed per-browser threshold is guaranteed by the Web Crypto specification.",{"heading":83,"paragraphs":84},"4. Recommendation: choose the implementation by file size",[85],"1. Small and medium files: enforce a configurable product limit, then use `file.arrayBuffer()` and `crypto.subtle.digest('SHA-256', buffer)`. 2. Genuinely large files: use a vetted incremental SHA-256 JS/Wasm implementation inside a Web Worker, read chunks sequentially, and update one hash state. Never hash chunks independently and concatenate their digests. 3. Progress and cancellation: the main thread should receive only progress and the final digest and must be able to terminate the Worker. A Worker isolates CPU work but does not remove memory requirements.",{"heading":87,"paragraphs":88,"code":90},"5. Complete code: size-bounded native Web Crypto SHA-256",[89],"This code states the native API boundary honestly: it accepts files only below an explicit limit and reads the full input once. `64 MiB` is an example product policy, not a browser guarantee; calibrate it across supported devices. Route larger files to an incremental Worker implementation.","const DEFAULT_NATIVE_DIGEST_LIMIT = 64 * 1024 * 1024;\n\nasync function computeFileSha256WithWebCrypto(\n  file: File,\n  maxBytes = DEFAULT_NATIVE_DIGEST_LIMIT,\n): Promise\u003Cstring> {\n  if (file.size > maxBytes) {\n    throw new RangeError(\n      \"File exceeds the native digest memory policy; use incremental SHA-256 in a Worker\",\n    );\n  }\n\n  const bytes = await file.arrayBuffer();\n  const digest = await crypto.subtle.digest(\"SHA-256\", bytes);\n  return Array.from(new Uint8Array(digest), (byte) =>\n    byte.toString(16).padStart(2, \"0\"),\n  ).join(\"\");\n}",{"heading":92,"paragraphs":93},"6. Incorrect approaches",[94],"Calling `SubtleCrypto.digest('MD5', buffer)` expecting MD5, resulting in unhandled DOMException errors. Blocking the UI main thread with endless synchronous hash loops. Loading entire multi-gigabyte files without checking size bounds.",{"heading":96,"paragraphs":97},"7. Boundaries: device variance, Workers, and buffer ownership",[98],"Do not rely on a fixed memory threshold that no browser specification guarantees; available memory varies by device, browser version, and concurrent tabs. Transfer chunk buffers to the Worker to reduce copying, but verify that the incremental library does not retain every chunk. Cancellation, navigation, and Worker errors must release resources.",{"heading":100,"paragraphs":101},"8. Verification",[102],"Compare browser output with `shasum -a 256` or a server-side SHA-256 implementation. Fix test vectors for empty files, non-aligned sizes, zero bytes, and over-limit inputs. For the Worker path, measure peak memory, main-thread long tasks, cancellation behavior, and monotonic progress across file sizes.",{"heading":104,"paragraphs":105},"9. FAQ",[106,107,108],"Can I call `subtle.digest()` on every `Blob.slice()` and concatenate the results? No. That produces independent digests and is not the SHA-256 of the complete file.","How should a 2GB file be handled? Use a trusted incremental SHA-256 JS/Wasm implementation in a Worker, feeding chunks into one state while validating supply chain, licensing, and standard test vectors.","Does a Worker prevent OOM? No. It isolates computation, but you must still bound chunks, avoid retaining all input, and measure peak memory.",{"heading":110,"paragraphs":111},"10. Summary",[112],"Native Web Crypto `digest()` is non-streaming. Use it only below an explicit memory policy; for large files, use a Worker plus a vetted incremental hash implementation and verify it with standard vectors and resource measurements. Empty chunk loops are not streaming SHA-256.",{"name":114,"url":115},"Tiny's Tool Editorial Team","https://tinystool.org/about",{"name":117,"url":115},"Tiny's Tool Technical Review","2026-08-08",[120],{"title":121,"url":122,"publisher":123},"Web Cryptography Level 2","https://www.w3.org/TR/WebCryptoAPI/","W3C",false,[126,143,159],{"slug":127,"noIndex":128,"category":129,"date":7,"updatedDate":118,"reviewedDate":118,"readingMinutes":130,"tags":131,"relatedTool":14,"cn":137,"en":140},"hash-md5-charset-utf8-gbk-mismatch",true,"踩坑避坑",6,[132,133,134,135,136],"MD5","Hash","Charset","UTF-8","GBK",{"title":138,"description":139},"为什么前后端算出来的 MD5 不一致？字符集编码 (UTF-8 vs GBK) 隐形坑","剖析哈希函数作用于底层字节流而非抽象字符的本质，分析 UTF-8 与 GBK 编码下相同字符串物理字节差异导致的 MD5 不一致问题与校验对齐方案。",{"title":141,"description":142},"Why Front-End and Back-End MD5 Hashes Differ: UTF-8 vs. GBK Encoding Mismatches","Explain how hash functions process raw byte streams rather than abstract characters, detailing UTF-8 vs. GBK encoding mismatches and resolution strategies.",{"slug":144,"noIndex":124,"category":145,"date":7,"updatedDate":118,"reviewedDate":118,"readingMinutes":146,"tags":147,"relatedTool":14,"cn":153,"en":156},"hash-pbkdf2-salting-sha256-password-security","最佳实践",8,[148,149,150,151,152],"Salt","PBKDF2","SHA-256","Password Security","Cryptography",{"title":154,"description":155},"加盐 (Salt) 与 PBKDF2：为什么单纯用 SHA-256 存储用户密码是不安全的","解析彩虹表与 GPU 暴力破解密码机制，说明加盐与 PBKDF2 (HMAC-SHA256) 迭代慢哈希原理，并给出 Web Crypto API 密码哈希与验证代码。",{"title":157,"description":158},"Salting and PBKDF2: Why Plain Single-Iteration SHA-256 Password Hashing Is Unsafe","Examine rainbow table lookups and GPU brute-force mechanics, detailing random salt and PBKDF2 (HMAC-SHA256) slow hashing using the Web Crypto API.",{"slug":160,"noIndex":128,"category":6,"date":7,"updatedDate":7,"reviewedDate":118,"readingMinutes":161,"tags":162,"relatedTool":166,"cn":167,"en":170},"js-obfuscator-control-flow-flattening-dead-code",9,[163,164,165,13],"JS Obfuscator","Control Flow Flattening","Dead Code","/tools/dev/js-obfuscator",{"title":168,"description":169},"JavaScript 控制流平坦化与死代码注入原理：性能、可读性与调试成本","剖析控制流平坦化 (Control Flow Flattening) 开关、Dispatcher 分发循环与死代码注入原理，评估对 CPU 执行开销、可读性与调试的副作用。",{"title":171,"description":172},"JavaScript Control Flow Flattening and Dead Code Injection: Trade-offs in Performance and Readability","Analyze Control Flow Flattening dispatchers and Dead Code Injection mechanisms, evaluating impacts on CPU execution overhead, bundle size, and debugging.",1786725508338]