staff
1,124
edits
(Added Avdump2 on Linux paragraph) |
({{Avdump-current-version}}) |
||
(79 intermediate revisions by 14 users not shown) | |||
Line 1: | Line 1: | ||
Welcome to the remarkably sparse avdump2 wiki page, lovingly hand created over many minutes. | {{TOCright}} | ||
{{eyecatch|Warning|Avdump2 is deprecated; please use {{Avdump-current-version}}.}} | |||
{{eyecatch|Important|Avdump2 fully replaces [[Avdump]], which is now deprecated and should not be used.}} | |||
{{eyecatch|Important|[[Avdump_GUI|Avdump GUI]] is a GUI version of Avdump2, for command line skeptics.}} | |||
Welcome to the remarkably sparse avdump2 wiki page, lovingly hand created over many minutes + some more. | |||
We gleefully embrace build | We gleefully embrace build 7100, yet another public release of avd2. It runs against a new build of MediaInfoLibrary (18.12), contains a number of custom solutions for a number of formats, represents a significant investment of time and energy by Arokh, and is just generally nifty. | ||
There is no general need to reprocess existing files with this build, but we don't mind if you do it anyway! | |||
==What is it== | ==What is it== | ||
'''Avdump2''' is a video codec identification utility - identifies which video codec and audio compression method is used on video files. Furthermore it calculates the used bit rates, the length, identifies the number of used audio channels and more.<br> | '''Avdump2''' is a video/audio codec identification utility - identifies which video codec and audio compression method is used on video files. Furthermore it hashes the file with a set of common hashalgorithms and calculates the used bit rates, the length, identifies the number of used audio channels and more.<br> | ||
It, like its predecessor, is mainly based on [http://mediainfo.sourceforge.net/en mediainfo], but significantly enhanced and changed to fit AniDB's needs by [[User:Arokh|Arokh]].<br> | It, like its predecessor, is mainly based on [http://mediainfo.sourceforge.net/en mediainfo], but significantly enhanced and changed to fit AniDB's needs by [[User:Arokh|Arokh]].<br> | ||
Line 13: | Line 17: | ||
* '''Video files:''' asf/wmv, avi, flv, m2ts, mk3d, mkv, mov, mp4, mpg/mpeg, ogm, ogv, qt, rm/rmvb, <u>swf</u>, ts, webm | * '''Video files:''' asf/wmv, avi, flv, m2ts, mk3d, mkv, mov, mp4, mpg/mpeg, ogm, ogv, qt, rm/rmvb, <u>swf</u>, ts, webm | ||
* '''Subtitle files:''' ass, idx, js, lrc, mks, pjs, rt, smi, srt, ssa, sub, tmp, tts, txt, xss | * '''Subtitle files:''' ass, idx, js, lrc, mks, pjs, rt, smi, srt, ssa, sub, sup, tmp, tts, txt, xss | ||
* '''Audio files:''' aac, ac3, dts, dtshd, flac, m4a, mka, mp3, ogg, ra, thd, wav, wma | * '''Audio files:''' aac, ac3, dts, dtshd, flac, m4a, mka, mp3, ogg, ra, thd, wav, wma | ||
* '''Archive files:''' <u>7z</u>, <u>ace</u>, <u>rar</u>, <u>zip</u> | * '''Archive files:''' <u>7z</u>, <u>ace</u>, <u>rar</u>, <u>zip</u> | ||
Line 19: | Line 23: | ||
===Download=== | ===Download=== | ||
[https://cdn.anidb.net/client/avdump2/avdump2_7100.zip DOWNLOAD(ZIP)] (Build 7100) MD5 = a1539a6139315a3c0f24113bb2d2853b | |||
[https://cdn.anidb.net/client/avdump2/avdump2_7101.zip DOWNLOAD(ZIP)] (Build 7101) MD5 = 2ba27f6477e8f933b366da65a6a8fdc2 | |||
===Usage=== | |||
'''Avdump2''' is a simple [[Wikipedia:Command line interface|command line]] client: | |||
:<tt>AVDump2CL.exe [--<Arg> -<OneLetterArg> ...] <media file/folder> [<media file/folder> ...]</tt> | |||
While pre 4991 build arguments can still be used, it is recommended to use the new format shown below. | |||
The easiest way to convert them to the new format is to run the program with the old args which then displays a notice and the corresponding args in the new format. | |||
====Options==== | ====Options==== | ||
Line 34: | Line 41: | ||
! style="border: 1px solid #aaaaaa; background: #f2f2f2;" | Example | ! style="border: 1px solid #aaaaaa; background: #f2f2f2;" | Example | ||
|- | |- | ||
|''' | |'''Auth'''=<username>:<udp_api_key>||Auto-Creqing||Sends file xml metadata dumps to the server for automatic creqing. You should always use this option when dumping anime files! Also enables error reporting.||||--Auth=tard:monkey | ||
|- | |- | ||
|''' | |'''Host'''=<hostname>:<port>||Auto-Creqing||Hostname of AniDB UDP API server||--Host=api.anidb.info:9002|| | ||
|- | |- | ||
|''' | |'''LPort'''=<port>||Auto-Creqing||Local UDP port used for autocreqing. Only needed when behind "strict" firewalls.||||--LPort=12345 | ||
|- | |- | ||
|''' | |'''TOut'''=<timeoutInSeconds>:<numOfRetries>||Auto-Creqing||Timeout for sending dumps||--TOut=10:3||--TOut=25:6 | ||
|- | |- | ||
| | | | ||
|- | |- | ||
|''' | |'''ACErr'''=<file path/name>||Logging||Write file paths of failed dumps to file.||||--ACErr=acerr.txt | ||
|- | |- | ||
|''' | |'''CRCErr'''=<file path/name>||Logging||Checks if filename contains calculated crc||||--CRCErr=crcerr.txt | ||
|- | |- | ||
|''' | |'''ExtDiff'''=<file path/name>||Logging||Write mismatched extensions and file paths to file.||||--ExtDiff=extdiff.txt | ||
|- | |- | ||
| | |'''FixExt'''=<file path/name>||Logging||Changes the fileextension to the dectected extension and logs the action.||||--FixExt=fixext.txt | ||
|- | |- | ||
|''' | |'''Done'''=<file path/name>||Logging||Skip paths present in log and append paths to newly processed files.||||--Done=done.txt | ||
|- | |- | ||
|''' | |'''HLog'''=<file path/name>||Logging||Export hashes to file.||||--HLog="$CRC$ $ED2K$":hlog.txt | ||
|- | |- | ||
|''' | |'''Log'''=<file path/name>||Logging||Write output to file.||||--Log=log.txt | ||
|- | |- | ||
| | | | ||
|- | |- | ||
|''' | |'''M'''||Output||MediaInfo dump||||-M | ||
|- | |- | ||
|''' | |'''m'''||Output||MediaInfo XML dump||||-m | ||
|- | |- | ||
|''' | |'''t'''||Output||Simple text format||||-t | ||
|- | |- | ||
| | |'''C'''||Output||XML creq format||||-C | ||
|- | |- | ||
|''' | |'''c'''||Output||AVD2 creq dump||||-c | ||
|- | |- | ||
|''' | |'''h'''||Output||Hash Dump||||-h | ||
|- | |- | ||
| | | | ||
|- | |- | ||
|''' | |'''BSize'''=<KB_per_block>:<number_of_blocks>||Control||Buffer size for hashing.||||--BSize=512:16 | ||
|- | |- | ||
|''' | |'''Ext'''=<comma separated list>||Control||Comma separated extension list. Prepend with "-" to exclude. ||Process all supported extensions||--Ext=avi,mkv or --Ext=-zip,-sub or --Ext=* | ||
|- | |- | ||
| | |'''Mon'''=<seconds>||Control||Monitor sleep duration in seconds.||--Mon=60||--Mon=120 | ||
|- | |- | ||
|''' | |'''NoErr'''||Control||Disable error reporting. You should not disable this as it helps finding and fixing bugs.||||--NoErr | ||
|- | |- | ||
|''' | |'''R'''||Control||Do '''not''' recurse into subfolders.||||-R | ||
|- | |- | ||
|'''p'''||Control||Pause when done (hold cmd window).||||-p | |'''p'''||Control||Pause when done (hold cmd window).||||-p | ||
|- | |- | ||
|''' | |'''P'''||Control||Pause after each file.||||-P | ||
|- | |- | ||
|'''r'''||Control||Random file order.||||-r | |'''r'''||Control||Random file order.||||-r | ||
|- | |- | ||
|''' | |'''q'''||Control||Suppress progress (silent).||||-q | ||
|- | |- | ||
|''' | |'''B'''||Control||Test pure hashingspeed without io limitations||||-B | ||
|- | |- | ||
|''' | |'''U'''||Control||Use UTF8 as console output stream||||-U | ||
|- | |- | ||
| | | | ||
Line 120: | Line 121: | ||
|'''a'''||Hashing||All (available) hash algorithms.||||-a | |'''a'''||Hashing||All (available) hash algorithms.||||-a | ||
|- | |- | ||
|''' | | | ||
|- | |||
|'''Exp'''=<file path/name>||AniDB||Export ed2k-links to the specified file. Very useful when mass adding files.||||--Exp=ed2k.txt | |||
|- | |- | ||
|''' | |'''PrintEd2kLink'''||AniDB|| ||||--PrintEd2kLink | ||
|- | |- | ||
|''' | |'''PrintAniDBLink'''||AniDB|| ||||--PrintAniDBLink | ||
|- | |- | ||
|''' | |'''OpenAniDBLink'''||AniDB|| ||||--OpenAniDBLink | ||
|- | |- | ||
|} | |} | ||
====Windows Shortcut==== | |||
#Extract Avdump to a folder of choice. | |||
#:Example: <tt>"C:\Program Files\AVDump2\"</tt> | |||
#Right-click <tt>AVDump2CL.exe</tt>, select ''Create Shortcut''. | |||
#Right-click the shortcut, add options to the ''Shortcut - Target'' line. | |||
#:Example, normal usage: <tt>"C:\Program Files\AVDump2\AVDump2CL.exe" -p</tt> | |||
#:Example, auto-creq usage: <tt>"C:\Program Files\AVDump2\AVDump2CL.exe" --Auth=''myName'':''udp_api_key''</tt> | |||
#::You can also set other options from the list above if you wish to do so. | |||
#::A commandline option to export ED2K links to a file is by adding <tt>--Exp=''C:\Path\ed2k.txt''</tt>, etc. | |||
#::Note that you need to set an '''UDP API Key''' in your AniDB profile for auto-creqing. | |||
#:Optional: Change ''Run'' to ''Maximized''. | |||
#Drag and drop any media-file onto the shortcut. | |||
{{eyecatch|Note|'''The ''udp_api_key'' is not your user password!!! see [[Avdump#Auto-creqing|Auto-creqing]] .}} | |||
====Send to==== | |||
#Move the shortcut created above to the SendTo folder. | |||
#:Example, under XP: <tt>"C:\Documents and Settings\''Username''\SendTo"</tt> | |||
#:Example, under Vista: <tt>"C:\Users\''Username''\AppData\Roaming\Microsoft\Windows\SendTo"</tt> | |||
#::Where ''Username'' is your Windows Login-Name. | |||
#:Note that ''SendTo'' is a hidden folder. | |||
#Right-click any media-file, go to "Send To", choose <tt>AVDump2CL.exe</tt> | |||
#Get [[Creq|CReqing]] | |||
====MS-DOS Batch File==== | |||
#Extract Avdump to a folder of choice. | |||
#Create a MS-DOS batch file with the 2 lines below, and save it as ''filename.bat''. | |||
#<tt>@ECHO OFF</tt> | |||
#<tt>C:\Program Files\AVDump\AVDump2CL.exe %* --Exp=''C:\Path\ed2k.txt'' --Auth=''myName'':''udp_api_key''</tt> | |||
#Drag and drop any media-file onto the shortcut. | |||
{{eyecatch|Note|'''The ''udp_api_key'' is not your user password!!! see [[Avdump#Auto-creqing|Auto-creqing]] .}} | |||
==Auto-creqing== | |||
[[Image:Autocreq.gif|thumb|Simple diagram of the way auto-creqing with Avdump2 works]] | |||
Avdump2 is the AniDB [[auto-creqing]] system. Some more or less important notes: | |||
* To be able to use this feature you’ll need an AniDB account '''and you have to define the ''UDP API Key'' in your [[profile]]'''. | |||
* All data sent to the server will be logged with IP and uid. | |||
* There is no direct connection between data sent to AniDB and creqs generated. The data received will just be stored for later processing. | |||
* There is no way to check the current status for a dump. Usually, it should take ''at least 24 hours'' from the moment you dump a file till the data actually changes. If any irregularities occur, or when there is too much data pending, it will take more time. | |||
* You may dump files currently not in the database. The data is still stored and will be used later if/after the file has been registered. | |||
* The creqs generated will report the user who sent the data first (for a specific file) as the creqer. | |||
* It is possible to run more than one instance of Avdump2 at once, up to a maximum of ten. At the moment, ''only 10 sessions per ip are allowed by the server'', meaning the eleventh instance will terminate the first. | |||
* Files creqed by the new system will be locked, meaning some fields will not be possible to change. Notify a moderator if you are '''sure''' that some of the data locked for a specific file is wrong. | |||
===Why didn't this file get dumped?=== | |||
* The file in AniDB is registered with wrong size and/or ED2K hash. | |||
* The package never reached the server. | |||
* The decryption or decompression failed at server side (rare). | |||
* The dump is not valid XML (rare). | |||
===Why isn't this file verified?=== | |||
Even if the file is dumped it doesn't mean it will get auto-creqed (and verified). Here are the reasons: | |||
* The file is corrupt/invalid ([http://anidb.net/perl-bin/animedb.pl?show=avmf&do=xml&avid=76226 example]). | |||
* The provided data is considered "incoherent". (Indicates a bug in Avdump.) | |||
* The dump was marked unfit for file verification by an AniDB Moderator. | |||
==Avdump2 on Linux== | |||
Thanks to Mono, Avdump2 runs natively on Linux as well. However, as most development effort goes into the Windows version and due to the heterogenity of Linux distributions, running Avdump2 on Linux isn’t as straightforward as running it on Windows. All Linux-related usage notes can be found on the [[Avdump2 on Linux]] page. | |||
==Changes From Avdump Classic== | ==Changes From Avdump Classic== | ||
Line 144: | Line 207: | ||
<Information> | <Information> | ||
<AVDump2CLVersion>0.1.3734.0</AVDump2CLVersion> | <AVDump2CLVersion>0.1.3734.0</AVDump2CLVersion> | ||
<RunOnBitness>64</RunOnBitness> | |||
<OS>Microsoft Windows NT 6.1.7601 Service Pack 1</OS> | |||
</Information> | </Information> | ||
<Exceptions> | <Exceptions> | ||
Line 165: | Line 230: | ||
</FileExceptions> | </FileExceptions> | ||
===Avdump2 | ==Known issues== | ||
* In some cases MediaInfoLib (MIL) used by Avdump2 might report an audio stream to have 0 or 3 channels. | |||
* Anamorphic .asf/.wmv and .mpg/.mpeg files might not get detected correctly and will report the PAR instead of the DAR. | |||
* Some corrupted mkv files may throw a (Thread Aborted) exception, this will be fixed once the updated version of the mkv parser is introduced | |||
==Changelog== | |||
<div style="height: 280px; overflow: auto; padding: 3px; border:1px solid #AAAAAA; font-size: small"> | |||
'''1.0.7101.0 : 2020.05.16''' | |||
* Allow compressed dumps of up to 14kb | |||
'''1.0.7100.0 : 2019.04.07''' | |||
* Don't stop execution on big endian systems | |||
* Warn when --ACreqErr is set but not --Auth | |||
* Don't report DisplayBuffer, LC_ALL, ThreadCreation failed and console resize exceptions | |||
* Removed annoying error message when --Auth is not enabled | |||
* Fixed MIL Versionstring | |||
* MIL Chapter workaround | |||
'''1.0.7080.0 : 2019.01.08''' | |||
* Removed thrownOn Attribute from Exception reports | |||
* Write ACreqErrors to Console | |||
* Filter Exceptions which should be reported | |||
* Donelog search is now case insensitive | |||
* Fix for Matroskafiles when the tracks sections is at the end of the file | |||
* Jaco Subtitle detection is now less trigger happy | |||
'''1.0.7070.0 : 2019.01.02''' | |||
* MIL Updated to Version 18.12 | |||
* Removed Client side MIL hash check | |||
* Skip logging/acreqing for files with errors (except for matroska files with header data) | |||
* EBML Parser (Matroska) crash fixed | |||
* Matroska Container Parser Update | |||
* Matroska Chapter Parsing | |||
* Matroska Parser Crash fixed (Missing Track and other corner cases) | |||
* Matroska: extra_size node added (Track overhead) | |||
* 3d flag | |||
* Added Mono Runtime Version to exception metadata | |||
* Display "Hashing failed" instead of nothing if hashing fails for whatever reason | |||
* Accept /? as parameter (same as --help) | |||
* Included MAC MIL library for mostly untested mac support | |||
* Added Dumpable extensions: trp | |||
* Switch --Version also prints MIL Version | |||
* Constrained TTH hash Thread to 4 instead of available corecount | |||
* MIL Info shuffling for consistency | |||
* Wait longer for MIL to return before hard crashing | |||
* Fixed -m Crash | |||
* MP4 Lag fixed / MIL Chapters bug workaround | |||
* Switch --NoMediaInfoProvider (switch is ignored when acreqing) | |||
* Don't eat cmd cursor on soft crash | |||
* Log acreq failure when preconditions aren't met | |||
* Added CodecFeatures node | |||
* Various fileextension provider fixes | |||
'''0.1.6714.0 : 2014.04.24''' | |||
* Check for invalid usernames (to skip check prepend @) | |||
* Less forgiving against corrupted files (i.e. older version may have creqed the files and this one doesn't) | |||
* Ogg/Ogm fix for corrupted files | |||
* Fixed matroska parser corner-case bug for large files | |||
'''0.1.6647.0 : 2013.03.29''' | |||
* Another attempt to fix OOM issues | |||
* Now any byte array used to read files is reused (=> reduces mem usage jumpiness) | |||
* Fixed non default blocksize/blockcount crashes | |||
* Fixed stackoverflow crashes on some old mono versions (TraceSource) | |||
* Added --Version argument (Prints the AVDump2 version) | |||
* Fixed some spelling errors | |||
* Added resynching for Matroska files (=> less exceptions for corrupted files) | |||
'''0.1.6525.0 : 2012.11.??''' | |||
* Reuse datablocks in an attempt to fix OOM issues on 32 bit linux (=> also more stable mem usage on all platforms) | |||
* Changed Naming: Password to API Key to avoid confusion (Ommina) | |||
'''0.1.6489.0 : 2012.10.??''' | |||
* Accept extensionless files when --Ext=* is used | |||
* Added disabled client exception | |||
* Fixed NullExcpetion bug (Invalid program hashes) | |||
'''0.1.6452.0 : 2012.09.??+x+y+z''' | |||
* Added notice when the test acreq had an invalid format | |||
'''0.1.6435.0 : 2012.09.??+x+y''' | |||
* Removed "log" extension from acreqable extensions | |||
* Added Framework Node in error reports | |||
* Added MIL filesize sanity check | |||
'''0.1.6361.0 : 2012.09.??+x''' | |||
* Changed BlockConsumer error message to state more clearly if the ACReq was sent or not | |||
* UTF8 for Error messages | |||
'''0.1.6359.0 : 2012.09.??''' | |||
* Moved to git repository | |||
* Changes lost in the ether | |||
'''0.1.6248.0 : 2012.08.28''' | |||
* Shiny icon for the executable | |||
* New AniDB Communication, should be faster and more reliable. | |||
** Big file reports no longer fail to be ACReqed. | |||
** Wrong Version and Wrong UserName/Password errors are now separated | |||
** Update notification will be shown if a newer version is available | |||
* InfoProvider: | |||
** Trimming all value fields | |||
** Fixes serverside parsing errors | |||
* FileExtensionProvider: | |||
** Limit ReadLine length | |||
** Fixes errors which can happen when a binary file is parsed | |||
* <strike>Directory: Added workaround for "too long" filepaths</strike> <br/> FileAdding: On error, display the path of the causing file/directory | |||
* MatroskaParser: | |||
** Less strict with corrupted mkv files. <br/> (If enough data could be read, the ACReq will be sent despite errors) | |||
* FileSource: Fixed nullpointer excpetion | |||
* CL: | |||
** Shortcut for DoneLogFileNameOnly is now D instead of R (because it clashed with NoRecurse) | |||
** Added version check and notification of new version if present | |||
** Less jumping for the ETA display | |||
* Added extensive tracing (removed in public versions) | |||
* Lots of other small things | |||
* | |||
'''0.1.5290.0 : 2012.05.12''' | |||
* | * Fixed FileExtensionParser (hopefully) | ||
* | * TXT Report should be fixed now | ||
* | * Fixed ACReq Send Thread crash | ||
* | * AVD2 no longer pauses when not on Windows | ||
* Fixed AVD2 not working when not started from same folder as the executable | |||
* Added switch --UseCWD | |||
* Other changes I haven't remembered yet | |||
'''0.1.5028.0 : 2011.09.11''' | |||
* Fixed shift of hashvalues (_Display_ only) | |||
* | * Fixed --Ext with "to remove" extensions | ||
* | * Fixed --Mon/--TOut to actually work | ||
* | * Fixed some bug in the old CLI converter | ||
* <no ponies> | |||
'''0.1.5000.0 : 2011.09.11''' | |||
* all previous builds aside of 4991 have been disabled | |||
* <more ponies> | |||
'''0.1.4964.0 : 2011.09.05''' | |||
* all previous builds have been disabled | |||
* Sending ACReqs asynchronous | |||
* <insert ponies riding down the rainbow and other stuff that has happened> | |||
'''0.1.4334.0 : 2011.06.15''' | '''0.1.4334.0 : 2011.06.15''' | ||
* all previous builds have been disabled | * all previous builds have been disabled |