Ed2k-hash: Difference between revisions

3,454 bytes added ,  19 December 2021
{{Avdump-current-version}}
mNo edit summary
({{Avdump-current-version}})
(14 intermediate revisions by 7 users not shown)
Line 1: Line 1:
{{TOCright}}
{{TOCright}}
== What's an ed2k hash? ==
== What's an ed2k hash? ==
In short, a hash is a checksum. It is used to detect errors in data by checking the integrity of the message/file. There are various different hash algorithms used for differing purposes, some that are focused more on security and some on speed. AniDB sports both kinds; CRC is not particularly secure at all, while MD5, SHA-1 and the MD4/ed2k hashes are much more secure from a cryptographic standpoint. They are not failsafe though, but having more than one hash makes it exponentially harder to counterfeit a file. You might be able to fool one of the hashes, but generally not two, or as is the case with most of the files found in AniDB: four hashes!
In short, a hash is a checksum. It is used to detect errors in data by checking the integrity of the message/file. There are various different hash algorithms used for differing purposes, some that are focused more on security and some on speed. AniDB sports both kinds; CRC is not particularly secure at all, while MD5, SHA-1 and the MD4/ed2k hashes are much more secure from a cryptographic standpoint. They are not fail-safe though, but having more than one hash makes it exponentially harder to counterfeit a file. You might be able to fool one of the hashes, but generally not two, or as is the case with most of the files found in AniDB: four hashes!


The ed2k hash is based on the ''md4 algorithm'', but rather than providing a single hash of the entire file, it breaks the file up into 9500kb ''chunks'' and produces a final hash based on the md4 sums of the chunks. While md4 longer considered secure from a cryptographic perspective, for the purpose of uniquely identifying files it's more than adequate. It is often listed as part of an ''ed2k link'', which also includes a size in bytes, and a name.
The ed2k hash is based on the ''md4 algorithm'', but rather than providing a single hash of the entire file, it breaks the file up into 9500kb ''chunks'' and produces a final hash based on the md4 sums of the chunks. While md4 is no longer considered secure from a cryptographic perspective, for the purpose of uniquely identifying files it's more than adequate. It is often listed as part of an ''ed2k link'', which also includes a size in bytes, and a name.


== Why does AniDB require ed2k-hashes? ==
== Why does AniDB require ed2k-hashes? ==
Line 20: Line 21:
== Which software can be used to generate them? ==
== Which software can be used to generate them? ==
If you have the file(s) on your hard disk or on CD you can use all kinds of tools to generate the ed2k-hash and other additional info:
If you have the file(s) on your hard disk or on CD you can use all kinds of tools to generate the ed2k-hash and other additional info:
* [[Avdump]] is the preferred tool as it can automatically creq files.
* {{Avdump-current-version}} is the preferred tool as it can automatically creq files.
* [[AniDB_O'Matic]] by BennieB/PetriW can generate ed2k-hashes. It generates ed2k-hashes, md5, sha-1, crc32 in one go and also lists stuff like codec, resolution, bitrates, ...
* [[AniDB O'Matic]] by BennieB/PetriW can generate ed2k-hashes. It generates ed2k-hashes, md5, sha-1, crc32 in one go and also lists stuff like codec, resolution, bitrates, ...
* [http://ed2k-tools.sourceforge.net/index.shtml ed2k_hash] another tool which is commandline based (current Windows and OSX versions have a GUI as well) and also available for Unix operating systems (Linux/BSD/etc).
* [http://ed2k-tools.sourceforge.net/index.shtml ed2k_hash] another tool which is commandline based (current Windows and OSX versions have a GUI as well) and also available for Unix operating systems (Linux/BSD/etc).
* [http://malich.content.no-ip.org/filehash/filehash-0.5.3.zip Filehash] a little Java program written by Malich. For further info on it read [http://forum.anidb.net/viewtopic.php?t=77 here]
* [http://malich.content.no-ip.org/filehash/filehash-0.5.3.zip Filehash] a little Java program written by Malich. For further info on it read {{OldThreadLink|77|here}}
* [http://www.slavasoft.com/hashcalc/ Hashcalc] can create md5, sha-1, crc32, ed2k-hashes and various other hashes in 1 go.
* [http://www.slavasoft.com/hashcalc/ Hashcalc] can create md5, sha-1, crc32, ed2k-hashes and various other hashes in 1 go.
* [http://rhash.anz.ru/ RHash] multiplatform open source console utility for computing md5/sha1/crc32 hashes, EDonkey and Magnet links for a directory tree.


== How is an ed2k hash calculated exactly? ==
== How is an ed2k hash calculated exactly? ==
A file is hashed in 9728000 byte ''chunks'', using the md4 algorithm, and produces a 128 bit hash for each chunk. For files with only one chunk, the ed2k hash ''is'' the md4 of the file, however for hashes with 2 or more chunks the the hash of each chunk is appended to those before it, and an further md4 of the hashes themselves provides the ed2k hash of the file. Pseudo code is given below:
A file is hashed in 9728000 byte ''chunks'', using the md4 algorithm, and produces a 128 bit hash for each chunk. For files with only one chunk, the ed2k hash ''is'' the md4 of the file, however for hashes with 2 or more chunks the the hash of each chunk is appended to those before it, and an further md4 of the hashes themselves provides the ed2k hash of the file. Pseudo code is given below:


:if filesize is less than <font color="blue">or equal to</font> 9728000:
:if filesize is less than {{colour|blue|or equal to}} 9728000:
:: return md4 of file
:: return md4 of file
:for chunk of size upto 9728000 in file:
:for chunk of size upto 9728000 in file:
:: append md4 of chunk to hashlist
:: append md4 of chunk to hashlist
:<font color="red">if filesize is a multiple of 9728000:
:{{colour|red|if filesize is a multiple of 9728000:}}
::append md4 of null to hashlist</font>
::{{colour|red|append md4 of null to hashlist}}
:return md4 of hashlist
:return md4 of hashlist


Line 40: Line 42:


=== List of which clients use which method ===
=== List of which clients use which method ===
[[Avdump]] use both methods. It will creq files from blue to red and store blue in file description ([http://anidb.net/f7047 Alternative ed2k]).
{{Avdump-current-version}} uses both methods. It will creq files from blue to red and store blue in file description ({{AniDBShortLink|f7047|Alternative ed2k}}).
*<font color="blue">edonkey2000 v0.5.0 to v1.4.3</font>
*{{colour|blue|edonkey2000 v0.5.0 to v1.4.3}}
*<font color="blue">mldonkey (2.5.30.17 tested)</font>
*{{colour|blue|mldonkey (2.5.30.17 tested)}}
*<font color="blue">shareaza (1.8 tested)</font>
*{{colour|blue|shareaza (1.8 tested)}}
*<font color="blue">HashCalc Version (2.01 tested)</font>
*{{colour|blue|HashCalc Version (2.01 tested)}}
*<font color="blue">edonkey-tool-hash (0.4.0 tested)</font>
*{{colour|blue|edonkey-tool-hash (0.4.0 tested)}}
*<font color="blue">fsum (2.51 tested)</font>
*{{colour|blue|fsum (2.51 tested)}}
*<font color="blue">ed2k_hash (0.4.0 tested)</font>
*{{colour|blue|ed2k_hash (0.4.0 tested)}}
*<font color="blue">hashgen (0.0.6 tested)</font>
*{{colour|blue|hashgen (0.0.6 tested)}}
*<font color="red">edonkey2000 until v0.5.0</font>
*{{colour|red|edonkey2000 until v0.5.0}}
*<font color="red">emule (0.46c tested)</font>
*{{colour|red|emule (0.46c tested)}}
*<font color="red">AOM (0.5.5.239 tested)</font>
*{{colour|red|AOM (0.5.5.239 tested)}}
*<font color="red">webaom (v1.13 tested)</font>
*{{colour|red|webaom (v1.13 tested)}}
*<font color="red">ed2k code by Stephane D'Alu (1.4 tested)</font>
*{{colour|red|ed2k code by Stephane D'Alu (1.4 tested)}}
*{{colour|red|jacksum (1.7.0 tested)}}
*{{colour|red|rhash (1.1.9 tested)}}


=== List of affected files, by fileID ===
=== List of affected files, by fileID ===
Line 66: Line 70:
|File of zeros || 19456000 || 194ee9e4fa79b2ee9f8829284c466051 || 114b21c63a74b6ca922291a11177dd5c
|File of zeros || 19456000 || 194ee9e4fa79b2ee9f8829284c466051 || 114b21c63a74b6ca922291a11177dd5c
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f7047 || 145920000 || 1c2b1a6b142955d84af5d3210d3ece6f || 4f79548623c6099896a489257163764e
|https://anidb.net/f7047 || 145920000 || 1c2b1a6b142955d84af5d3210d3ece6f || 4f79548623c6099896a489257163764e
|-
|https://anidb.net/f31383 || 136192000 || f869547f07275eda0067694540c9dc93 || df294338b38a29f81ad84f1f364b4504
|- style="background-color: #eee;"
|https://anidb.net/f48530 || 175104000 || a641926fa474fafe3f2c12676ea66b8e (?) || c110c2b684aaa391a980bde4e6ee9f1f
|-
|https://anidb.net/f51131 || 107008000 || 5911beead79e0fa9043baee70683ef56 || aa399ff3a0ab9f8eb939dbcd7b7d0ec3
|- style="background-color: #eee;"
|https://anidb.net/f51330 || 175104000 || fa7fbadaed151b003032985eae5c3420 || 148b2cf54cb4d66f70939ec5224d7961
|-
|https://anidb.net/f55744 ||  97280000 || 4c8a9540fe5aa2f4d9f8cac835a071a6 || 2fcd55bdeae2a92cc99d70763a64f048
|- style="background-color: #eee;"
|https://anidb.net/f56411 || 165376000 || b7b2f5eaea94bf89f5e03a775d9d9478 || f498072c0849cee180e4a1a7d34a26d2
|-
|https://anidb.net/f57766 || 145920000 || 7a54eda5d89ed525974487aa94515701 || 85d995b678284e7db5d52df1375971a9
|- style="background-color: #eee;"
|https://anidb.net/f73921 || 184832000 || f8fdbb017cd74ff66882a7c6f33fae24 || fc9210c307f99ed7339556d5f05f3d59
|-
|https://anidb.net/f78552 || 194560000 || 8f1fb4062cd1e8f578013e9b7719d05a || f93db3a2ed31e7f48fe35945e4c5c6e8
|- style="background-color: #eee;"
|https://anidb.net/f80216 || 165376000 || d61b705c59199666e164a274e7f91bec || ddcec8fdcddd43276a2c173498345789
|-
|https://anidb.net/f92884 ||  68096000 || c310997efade26c107e44036c1fd0dc3 || b83bdad42c5ae5204bea3a25959e2180
|- style="background-color: #eee;"
|https://anidb.net/f123554 || 243200000 || 822fc0f338fe8e43d96b9a99fe9632ce || ee3557fe68ccd056302710a185f4445f
|-
|https://anidb.net/f126410 || 184832000 || e19581a5518a11fbd50c1f23f9c21b95 || 87ac7a62de204473d3f5448214f4207c
|- style="background-color: #eee;"
|https://anidb.net/f130233 || 184832000 || 91ae6f6b3bb42e0792c63efb9f1aa81e (?) || 8ad09e1b46b695ebae0243b1856e801c
|-
|https://anidb.net/f142402 || 155648000 ||                                  || 4538e1fded9d7661e1ad7d56e7406054 (?)
|- style="background-color: #eee;"
|https://anidb.net/f165143 || 165376000 || 0ced631bb9010d3ccd331689a2fb02de (?) || 6a092c056bc46e7a08d63408f918ba52
|-
|-
|http://anidb.net/f24359 || 136192000 || f869547f07275eda0067694540c9dc93 || df294338b38a29f81ad84f1f364b4504
|https://anidb.net/f166304 || 184832000 || aab2ce19d5b786af20d6e4a15f63552f || aa9930ccd300a2feac30b0e49830c321
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f31383 || 136192000 || f869547f07275eda0067694540c9dc93 || df294338b38a29f81ad84f1f364b4504
|https://anidb.net/f174421 || 223744000 || cd87540a7b48e87e78e7c714fbf7581e || a7c9a857b6d584bae0568495b92ea609
|-
|-
|http://anidb.net/f48530 || 175104000 || a641926fa474fafe3f2c12676ea66b8e || c110c2b684aaa391a980bde4e6ee9f1f
|https://anidb.net/f200233 || 58368000 || cfff67163f6ca9bbed26211e140b10e1 || 0943c164b8a076c88551f0b2c1757436
 
|- style="background-color: #eee;"
|https://anidb.net/f220069 || 389120000 || a182915f5cd114937f760246da92234b || 43324839e08afca1d83a890468f87f5e
|-
|https://anidb.net/f240925 || 126464000 || bae493f413037066ce5a597be4d97e8f || 725a209cf70715b3fdfb03348bb040c6
|- style="background-color: #eee;"
|https://anidb.net/f243620 || 311296000 || 8d81479f7b5ba92c3094630899b5ec7a (?) || 1cc497d5f73d9365e88dcdaa4207559e
|-
|https://anidb.net/f245201 || 204288000 || cd3c10c268efda196dbe194164a90fbd || 2923a3dbc14257819fdbb4e8db3cf216
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f51131 || 107008000 || 5911beead79e0fa9043baee70683ef56 || aa399ff3a0ab9f8eb939dbcd7b7d0ec3
|https://anidb.net/f274687 || 126464000 ||                                 || eb10e928e0b3eb494201a15a9609e155 (?)
|-
|-
|http://anidb.net/f51330 || 175104000 || fa7fbadaed151b003032985eae5c3420 || 148b2cf54cb4d66f70939ec5224d7961
|https://anidb.net/f285297 || 204288000 || 152aee605ad7f52bd49caa3c6ec3391e || 9960cbb5ccc6708ae636847965c7613a
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f55744 || 97280000 || 4c8a9540fe5aa2f4d9f8cac835a071a6 || 2fcd55bdeae2a92cc99d70763a64f048
|https://anidb.net/f293772 || 204288000 ||                                 || e218f093e52e894904e30a90a889572f (?)
|-
|-
|http://anidb.net/f56411 || 165376000 || b7b2f5eaea94bf89f5e03a775d9d9478 || f498072c0849cee180e4a1a7d34a26d2
|https://anidb.net/f296905 || 136192000 || e6c79c25dbcf01f1766b5e11b3c5436d || 44e79c0e98cf82ae1b1262aa93f69831
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f57766 || 145920000 || 7a54eda5d89ed525974487aa94515701 || 85d995b678284e7db5d52df1375971a9
|https://anidb.net/f302567 || 68096000 ||                                 || 681c7efc40f9858599c6f2cce56d53cd
|-
|-
|http://anidb.net/f73921 || 184832000 || f8fdbb017cd74ff66882a7c6f33fae24 || fc9210c307f99ed7339556d5f05f3d59
|https://anidb.net/f323369 || 301568000 || a12fdc594ac4a74cabb7e36b28febd8f || b18a7f002491ab8eb626592440409bb1
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f78552 || 194560000 || 8f1fb4062cd1e8f578013e9b7719d05a || f93db3a2ed31e7f48fe35945e4c5c6e8
|https://anidb.net/f382969 || 398848000 || da98157df3db32cd6ef77f93daebda92 || 1dc31b64cc7fe8a163b681750da1bede
|-
|-
|http://anidb.net/f80216 || 165376000 || d61b705c59199666e164a274e7f91bec || ddcec8fdcddd43276a2c173498345789
|https://anidb.net/f387858 || 165376000 || 52719437d3966e25e7eea637bc05095d || 93d1ca5705d44c5f61c336017c4e0e91
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f92884 || 68096000 || c310997efade26c107e44036c1fd0dc3 || b83bdad42c5ae5204bea3a25959e2180
|https://anidb.net/f417095 || 243200000 || 49cc689241969de8d3c6a0e8f457e9eb || 3fca0bed5958054b5602d9b981b3b7c6
|-
|-
|http://anidb.net/f123554 || 243200000 || 822fc0f338fe8e43d96b9a99fe9632ce || ee3557fe68ccd056302710a185f4445f
|https://anidb.net/f461593 || 175104000 || e121090c50cda8d08f6db55dce26310b || 4d221b9696a86df3d9a4cfe36ed16d13
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f126410 || 184832000 || e19581a5518a11fbd50c1f23f9c21b95 || 87ac7a62de204473d3f5448214f4207c
|https://anidb.net/f462893 || 262656000 || 57fd0ca8153e8dc7d50f1c3137532165 || 3362c78e0a4ffcd2b52ceb164a150eb4
|-
|-
|http://anidb.net/f130233 || 184832000 || 91ae6f6b3bb42e0792c63efb9f1aa81e || 8ad09e1b46b695ebae0243b1856e801c
|https://anidb.net/f548516 || 175104000 || 31d9cb3408c295b2f8ab99e59dbc8415 || 386b6754dd8423a4a6c7758c321c3aa5
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f142402 || 155648000 || || 4538e1fded9d7661e1ad7d56e7406054
|https://anidb.net/f574589 || 252928000 || 6fce3c890fa508b60987e12f2659c195 || 6919048b7fc478d9d52d6af2a4a2aa09
|-
|-
|http://anidb.net/f165143 || 165376000 || 0ced631bb9010d3ccd331689a2fb02de || 6a092c056bc46e7a08d63408f918ba52
|https://anidb.net/f650373 || 272384000 || 6ad5aa68b4ce0d1146f4961e6dd7463f || e7af8a83a42b40f1ac5a5faf717baf55
|- style="background-color: #eee;"
|- style="background-color: #eee;"
|http://anidb.net/f166304 || 184832000 || aab2ce19d5b786af20d6e4a15f63552f || aa9930ccd300a2feac30b0e49830c321
|https://anidb.net/f669759 || 214016000 || 9e5bdeeb1f96b1e10b98ee488217089f || 469181b3481f85d6aba79ed97d5d9f07
|-
|-
|http://anidb.net/f174421 || 223744000 || cd87540a7b48e87e78e7c714fbf7581e || a7c9a857b6d584bae0568495b92ea609
|https://anidb.net/f761921 || 252928000 || 915c361308b30b2482148410cbb1ab99 || 7327220ef6a049a876f430a25ba498d6
|- style="background-color: #eee;"
|https://anidb.net/f819528 || 243200000 || 569681cc21e619f60c9320ada33d3e9e || 2c6f0b959a563fd56fe9e2f4e8bb66db
|-
|-
|http://anidb.net/f200233 || 58368000 || cfff67163f6ca9bbed26211e140b10e1 || 0943c164b8a076c88551f0b2c1757436
|https://anidb.net/f823379 || 214016000 || e00a2b09e0766b3965b2c254f6801f06 || 262be6b7ab3b0bb3bddd1fde83e711a4
|- style="background-color: #eee;"
|https://anidb.net/f870957 || 116736000 || 8a12925b6bbca3963aa25da9018e8bba || 556776c27ec944e85bffd69a7946e70b
|-
|-
|http://anidb.net/f220069 || 389120000 || a182915f5cd114937f760246da92234b || 43324839e08afca1d83a890468f87f5e
|https://anidb.net/f942702 || 340480000 || 64facbbd2cc4fff177926823f3fe9a6b || 9d5521ca84e45ee12dacda29a0c9a333
|- style="background-color: #eee;"
|https://anidb.net/f1005498 || 175104000 || 992f78c5fe3dc55d074e3970d1ae8a6b || 00f1b6c7847b7ca651297ee8f693f083
|-
|-
|http://anidb.net/f240925 || 126464000 || bae493f413037066ce5a597be4d97e8f || 725a209cf70715b3fdfb03348bb040c6
|https://anidb.net/f1218460 || 233472000 || 178fdaf12852837cfdccd96acd2c7f89 || 1e026cae1ce9d72fe272b08313e6a6e7
|- style="background-color: #eee;"
|https://anidb.net/f1273642 || 214016000 || 40b5772aaa58d3652caa40c97ba0de64 || 008356fd9a39e30872ccf3c62074bf5a
|-
|-
|http://anidb.net/f243620 || 311296000 || 8d81479f7b5ba92c3094630899b5ec7a || 1cc497d5f73d9365e88dcdaa4207559e
|https://anidb.net/f1681919 || 272384000 ||                                 || f663dbeb93069df2b1b5b27cacafc579 (?)
|}
|}


[[Category: Definitions]][[Category: Guidelines]]
==See also==
*[[Export ED2K Links]]
[[Category:Definitions]][[Category:Guidelines]]
staff
1,124

edits

MediaWiki spam blocked by CleanTalk.
MediaWiki spam blocked by CleanTalk.