Avdump2: Difference between revisions

6,985 bytes added ,  19 December 2021
{{Avdump-current-version}}
No edit summary
({{Avdump-current-version}})
 
(69 intermediate revisions by 13 users not shown)
Line 1: Line 1:
{{TOCright}}
{{eyecatch|Warning|Avdump2 is deprecated; please use {{Avdump-current-version}}.}}
{{eyecatch|Important|Avdump2 fully replaces [[Avdump]], which is now deprecated and should not be used.}}
{{eyecatch|Important|[[Avdump_GUI|Avdump GUI]] is a GUI version of Avdump2, for command line skeptics.}}
Welcome to the remarkably sparse avdump2 wiki page, lovingly hand created over many minutes + some more.
Welcome to the remarkably sparse avdump2 wiki page, lovingly hand created over many minutes + some more.


We gleefully embrace build 5028, yet another public release of avd2. It represents a complete rewrite of the original avd1 code, runs against a new build of MediaInfoLibrary, contains a number of custom solutions for a number of formats, represents a significant investment of time and energy by Arokh, and is just generally nifty.
We gleefully embrace build 7100, yet another public release of avd2. It runs against a new build of MediaInfoLibrary (18.12), contains a number of custom solutions for a number of formats, represents a significant investment of time and energy by Arokh, and is just generally nifty.


This release contains a minor collection of fixes from previous releases, and, barring discovery of world-ending bugs in the interim, will be the first version to be formally announced on the web site proper.  As before, there is no need to reprocess existing files with this build, but do keep an eye out for misbehaviour.
There is no general need to reprocess existing files with this build, but we don't mind if you do it anyway!


==What is it==
==What is it==
Line 19: Line 23:


===Download===
===Download===
 
[https://cdn.anidb.net/client/avdump2/avdump2_7100.zip DOWNLOAD(ZIP)] (Build 7100) MD5 = a1539a6139315a3c0f24113bb2d2853b
  [http://static.anidb.net/client/avdump2/avdump2_5000.rar DOWNLOAD(RAR)] (Build 5000) MD5 = <hell knows>
  [https://cdn.anidb.net/client/avdump2/avdump2_7101.zip DOWNLOAD(ZIP)] (Build 7101) MD5 = 2ba27f6477e8f933b366da65a6a8fdc2


===Usage===
===Usage===
'''Avdump''' is a simple [[Wikipedia:Command line interface|command line]] client:
'''Avdump2''' is a simple [[Wikipedia:Command line interface|command line]] client:
:<tt>AVDump2CL.exe [--<Arg> -<OneLetterArg> ...] <media file/folder> [<media file/folder> ...]</tt>
:<tt>AVDump2CL.exe [--<Arg> -<OneLetterArg> ...] <media file/folder> [<media file/folder> ...]</tt>


Line 37: Line 41:
! style="border: 1px solid #aaaaaa; background: #f2f2f2;" | Example
! style="border: 1px solid #aaaaaa; background: #f2f2f2;" | Example
|-
|-
|'''Auth'''=<username>:<udp_api_key>||Auto-Creqing||Enable automatic creqing. You should always use this option when dumping anime files! Also enables error reporting.||||--Auth:tard:monkey
|'''Auth'''=<username>:<udp_api_key>||Auto-Creqing||Sends file xml metadata dumps to the server for automatic creqing. You should always use this option when dumping anime files! Also enables error reporting.||||--Auth=tard:monkey
|-
|-
|'''Host'''=<hostname>:<port>||Auto-Creqing||Hostname of AniDB UDP API server||--Host=api.anidb.info:9002||
|'''Host'''=<hostname>:<port>||Auto-Creqing||Hostname of AniDB UDP API server||--Host=api.anidb.info:9002||
Line 43: Line 47:
|'''LPort'''=<port>||Auto-Creqing||Local UDP port used for autocreqing. Only needed when behind "strict" firewalls.||||--LPort=12345
|'''LPort'''=<port>||Auto-Creqing||Local UDP port used for autocreqing. Only needed when behind "strict" firewalls.||||--LPort=12345
|-
|-
|'''TOut'''=<timeoutInSeconds>:<numOfRetries>||Auto-Creqing||Timeout for sending dumps||--TOut:10:3||--TOut=25:6
|'''TOut'''=<timeoutInSeconds>:<numOfRetries>||Auto-Creqing||Timeout for sending dumps||--TOut=10:3||--TOut=25:6
|-
|-
| &nbsp;
| &nbsp;
|-
|-
|'''ACRrr'''=<file path/name>||Logging||Write file paths of failed dumps to file.||||--ACErr=acerr.txt
|'''ACErr'''=<file path/name>||Logging||Write file paths of failed dumps to file.||||--ACErr=acerr.txt
|-
|-
|'''CRCErr'''=<file path/name>||Logging||Checks if filename contains calculated crc||||--CRCErr=crcerr.txt
|'''CRCErr'''=<file path/name>||Logging||Checks if filename contains calculated crc||||--CRCErr=crcerr.txt
Line 79: Line 83:
|'''BSize'''=<KB_per_block>:<number_of_blocks>||Control||Buffer size for hashing.||||--BSize=512:16
|'''BSize'''=<KB_per_block>:<number_of_blocks>||Control||Buffer size for hashing.||||--BSize=512:16
|-
|-
|'''Ext'''=<comma separated list>||Control||Comma separated extension list.||Process all supported extensions||--Ext=avi,mkv or --Ext=-zip,-sub or --Ext=*
|'''Ext'''=<comma separated list>||Control||Comma separated extension list. Prepend with "-" to exclude. ||Process all supported extensions||--Ext=avi,mkv or --Ext=-zip,-sub or --Ext=*
|-
|-
|'''Mon'''=<seconds>||Control||Monitor sleep duration in seconds.||--Mon=60||--Mon=120
|'''Mon'''=<seconds>||Control||Monitor sleep duration in seconds.||--Mon=60||--Mon=120
Line 127: Line 131:
|'''OpenAniDBLink'''||AniDB|| ||||--OpenAniDBLink
|'''OpenAniDBLink'''||AniDB|| ||||--OpenAniDBLink
|-
|-
|}
====Windows Shortcut====
#Extract Avdump to a folder of choice.
#:Example: <tt>"C:\Program Files\AVDump2\"</tt>
#Right-click <tt>AVDump2CL.exe</tt>, select ''Create Shortcut''.
#Right-click the shortcut, add options to the ''Shortcut - Target'' line.
#:Example, normal usage: <tt>"C:\Program Files\AVDump2\AVDump2CL.exe" -p</tt>
#:Example, auto-creq usage: <tt>"C:\Program Files\AVDump2\AVDump2CL.exe" --Auth=''myName'':''udp_api_key''</tt>
#::You can also set other options from the list above if you wish to do so.
#::A commandline option to export ED2K links to a file is by adding <tt>--Exp=''C:\Path\ed2k.txt''</tt>, etc.
#::Note that you need to set an '''UDP API Key''' in your AniDB profile for auto-creqing.
#:Optional: Change ''Run'' to ''Maximized''.
#Drag and drop any media-file onto the shortcut.
{{eyecatch|Note|'''The ''udp_api_key'' is not your user password!!! see [[Avdump#Auto-creqing|Auto-creqing]] .}}
====Send to====
#Move the shortcut created above to the SendTo folder.
#:Example, under XP: <tt>"C:\Documents and Settings\''Username''\SendTo"</tt>
#:Example, under Vista: <tt>"C:\Users\''Username''\AppData\Roaming\Microsoft\Windows\SendTo"</tt>
#::Where ''Username'' is your Windows Login-Name.
#:Note that ''SendTo'' is a hidden folder.
#Right-click any media-file, go to "Send To", choose <tt>AVDump2CL.exe</tt>
#Get [[Creq|CReqing]]
====MS-DOS Batch File====
#Extract Avdump to a folder of choice.
#Create a MS-DOS batch file with the 2 lines below, and save it as ''filename.bat''.
#<tt>@ECHO OFF</tt>
#<tt>C:\Program Files\AVDump\AVDump2CL.exe %* --Exp=''C:\Path\ed2k.txt'' --Auth=''myName'':''udp_api_key''</tt>
#Drag and drop any media-file onto the shortcut.


|}
{{eyecatch|Note|'''The ''udp_api_key'' is not your user password!!! see [[Avdump#Auto-creqing|Auto-creqing]] .}}
 
==Auto-creqing==
[[Image:Autocreq.gif|thumb|Simple diagram of the way auto-creqing with Avdump2 works]]
 
Avdump2 is the AniDB [[auto-creqing]] system. Some more or less important notes:
* To be able to use this feature you’ll need an AniDB account '''and you have to define the ''UDP API Key'' in your [[profile]]'''.
* All data sent to the server will be logged with IP and uid.
* There is no direct connection between data sent to AniDB and creqs generated. The data received will just be stored for later processing.
* There is no way to check the current status for a dump. Usually, it should take ''at least 24 hours'' from the moment you dump a file till the data actually changes. If any irregularities occur, or when there is too much data pending, it will take more time.
* You may dump files currently not in the database. The data is still stored and will be used later if/after the file has been registered.
* The creqs generated will report the user who sent the data first (for a specific file) as the creqer.
* It is possible to run more than one instance of Avdump2 at once, up to a maximum of ten. At the moment, ''only 10 sessions per ip are allowed by the server'', meaning the eleventh instance will terminate the first.
* Files creqed by the new system will be locked, meaning some fields will not be possible to change. Notify a moderator if you are '''sure''' that some of the data locked for a specific file is wrong.
 
===Why didn't this file get dumped?===
* The file in AniDB is registered with wrong size and/or ED2K hash.
* The package never reached the server.
* The decryption or decompression failed at server side (rare).
* The dump is not valid XML (rare).
 
===Why isn't this file verified?===
Even if the file is dumped it doesn't mean it will get auto-creqed (and verified). Here are the reasons:
* The file is corrupt/invalid ([http://anidb.net/perl-bin/animedb.pl?show=avmf&do=xml&avid=76226 example]).
* The provided data is considered "incoherent". (Indicates a bug in Avdump.)
* The dump was marked unfit for file verification by an AniDB Moderator.
 
==Avdump2 on Linux==
Thanks to Mono, Avdump2 runs natively on Linux as well. However, as most development effort goes into the Windows version and due to the heterogenity of Linux distributions, running Avdump2 on Linux isn’t as straightforward as running it on Windows. All Linux-related usage notes can be found on the [[Avdump2 on Linux]] page.


==Changes From Avdump Classic==
==Changes From Avdump Classic==
Line 166: Line 230:
  </FileExceptions>
  </FileExceptions>


===Avdump2 on Linux===
==Known issues==
Thanks to Mono, Avdump2 runs natively on Linux as well. However, as most development effort goes into the Windows version and due to the heterogenity of Linux distributions, running Avdump2 on Linux isn’t as straightforward as running it on Windows. All Linux-related usage notes can be found on the [[Avdump2 on Linux]] page.
* In some cases MediaInfoLib (MIL) used by Avdump2 might report an audio stream to have 0 or 3 channels.
* Anamorphic .asf/.wmv and .mpg/.mpeg files might not get detected correctly and will report the PAR instead of the DAR.
* Some corrupted mkv files may throw a (Thread Aborted) exception, this will be fixed once the updated version of the mkv parser is introduced
 
==Changelog==
<div style="height: 280px; overflow: auto; padding: 3px; border:1px solid #AAAAAA; font-size: small">
'''1.0.7101.0 : 2020.05.16'''
* Allow compressed dumps of up to 14kb
 
'''1.0.7100.0 : 2019.04.07'''
* Don't stop execution on big endian systems
* Warn when --ACreqErr is set but not --Auth
* Don't report DisplayBuffer, LC_ALL, ThreadCreation failed and console resize  exceptions
* Removed annoying error message when --Auth is not enabled
* Fixed MIL Versionstring
* MIL Chapter workaround
 
'''1.0.7080.0 : 2019.01.08'''
* Removed thrownOn Attribute from Exception reports
* Write ACreqErrors to Console
* Filter Exceptions which should be reported
* Donelog search is now case insensitive
* Fix for Matroskafiles when the tracks sections is at the end of the file
* Jaco Subtitle detection is now less trigger happy
 
'''1.0.7070.0 : 2019.01.02'''
* MIL Updated to Version 18.12
* Removed Client side MIL hash check
* Skip logging/acreqing for files with errors (except for matroska files with header data)
* EBML Parser (Matroska) crash fixed
* Matroska Container Parser Update
* Matroska Chapter Parsing
* Matroska Parser Crash fixed (Missing Track and other corner cases)
* Matroska: extra_size node added (Track overhead)
* 3d flag
* Added Mono Runtime Version to exception metadata
* Display "Hashing failed" instead of nothing if hashing fails for whatever reason
* Accept /? as parameter (same as --help)
* Included MAC MIL library for mostly untested mac support
* Added Dumpable extensions: trp
* Switch --Version also prints MIL Version
* Constrained TTH hash Thread to 4 instead of available corecount
* MIL Info shuffling for consistency
* Wait longer for MIL to return before hard crashing
* Fixed -m Crash
* MP4 Lag fixed / MIL Chapters bug workaround
* Switch --NoMediaInfoProvider (switch is ignored when acreqing)
* Don't eat cmd cursor on soft crash
* Log acreq failure when preconditions aren't met
* Added CodecFeatures node
* Various fileextension provider fixes
 
 
'''0.1.6714.0 : 2014.04.24'''
* Check for invalid usernames (to skip check prepend @)
* Less forgiving against corrupted files (i.e. older version may have creqed the files and this one doesn't)
* Ogg/Ogm fix for corrupted files
* Fixed matroska parser corner-case bug for large files
 
'''0.1.6647.0 : 2013.03.29'''
* Another attempt to fix OOM issues
* Now any byte array used to read files is reused (=> reduces mem usage jumpiness)
* Fixed non default blocksize/blockcount crashes
* Fixed stackoverflow crashes on some old mono versions (TraceSource)
* Added --Version argument (Prints the AVDump2 version)
* Fixed some spelling errors
* Added resynching for Matroska files (=> less exceptions for corrupted files)
 
'''0.1.6525.0 : 2012.11.??'''
* Reuse datablocks in an attempt to fix OOM issues on 32 bit linux (=> also more stable mem usage on all platforms)
* Changed Naming: Password to API Key to avoid confusion (Ommina)
 
'''0.1.6489.0 : 2012.10.??'''
* Accept extensionless files when --Ext=* is used
* Added disabled client exception
* Fixed NullExcpetion bug (Invalid program hashes)
 
'''0.1.6452.0 : 2012.09.??+x+y+z'''
* Added notice when the test acreq had an invalid format
 
'''0.1.6435.0 : 2012.09.??+x+y'''
* Removed "log" extension from acreqable extensions
* Added Framework Node in error reports
* Added MIL filesize sanity check
 
'''0.1.6361.0 : 2012.09.??+x'''
* Changed BlockConsumer error message to state more clearly if the ACReq was sent or not
* UTF8 for Error messages
 
'''0.1.6359.0 : 2012.09.??'''
* Moved to git repository
* Changes lost in the ether
 
'''0.1.6248.0 : 2012.08.28'''
* Shiny icon for the executable
 
* New AniDB Communication, should be faster and more reliable.
** Big file reports no longer fail to be ACReqed.
** Wrong Version and Wrong UserName/Password errors are now separated
** Update notification will be shown if a newer version is available
 
* InfoProvider:
** Trimming all value fields
** Fixes serverside parsing errors
 
* FileExtensionProvider:
** Limit ReadLine length
** Fixes errors which can happen when a binary file is parsed


* <strike>Directory: Added workaround for "too long" filepaths</strike> <br/> FileAdding: On error, display the path of the causing file/directory


==Auto-creqing==
* MatroskaParser:
[[Image:Autocreq.gif|thumb|Simple diagram of the way auto-creqing with Avdump works]]
** Less strict with corrupted mkv files. <br/> (If enough data could be read, the ACReq will be sent despite errors)


Avdump2 is the AniDB [[auto-creqing]] system. Some more or less important notes:
* FileSource: Fixed nullpointer excpetion
* To be able to use this feature you’ll need an AniDB account '''and you have to define the ''UDP API Key'' in your [[profile]]'''. All data sent to the server will be logged with IP and uid.
* It is possible to run more than one instance of Avdump at once, but not more than ten will work OK. At the moment ''only 10 sessions per ip are allowed by the server'', meaning the eleventh instance will terminate the first.
* There is no direct connection between data sent to AniDB and creqs generated. The data received will just be stored for later processing. There is no way to check the current status for a dump. Usually, it should take ''at least 24 hours'' from the moment you dump a file till the data actually changes. If any irregularities occur, or when there is too much data pending, it will take more time.
* The creqs generated will report the user who sent the data first (for a specific file) as the creqer.
* You may dump files currently not in the database. The data is still stored and will be used later if/after the file has been registered.
* Files creqed by the new system will be locked, meaning some fields will not be possible to change. Notify a moderator if you are '''sure''' that some of the data locked for a specific file is wrong.


===Why didn't this file get dumped?===
* CL:
* The file in AniDB is registered with wrong size and/or ED2K hash.
** Shortcut for DoneLogFileNameOnly is now D instead of R (because it clashed with NoRecurse)
* The package never reached the server.
** Added version check and notification of new version if present
* The decryption or decompression failed at server side (rare).
** Less jumping for the ETA display
* The dump is not valid XML (rare).


===Why isn't this file verified?===
* Added extensive tracing (removed in public versions)
Even if the file is dumped it doesn't mean it will get auto-creqed (and verified). Here are the reasons:
* The file is corrupt/invalid ([http://anidb.net/perl-bin/animedb.pl?show=avmf&do=xml&avid=76226 example]).
* The provided data is considered "incoherent". (Indicates a bug in Avdump.)
* The dump was marked unfit for file verification by an AniDB Moderator.


* Lots of other small things


==Known issues==
'''0.1.5290.0 : 2012.05.12'''
* In some cases MediaInfoLib (MIL) used by Avdump2 might report an audio stream to have 0 or 3 channels.
* Fixed FileExtensionParser (hopefully)
* Anamorphic .asf/.wmv and .mpg/.mpeg files might not get detected correctly and will report the PAR instead of the DAR.
* TXT Report should be fixed now
* Some corrupted mkv files may throw a (Thread Aborted) exception, this will be fixed once the updated version of the mkv parser is introduced
* Fixed ACReq Send Thread crash
* AVD2 no longer pauses when not on Windows
* Fixed AVD2 not working when not started from same folder as the executable
* Added switch --UseCWD
* Other changes I haven't remembered yet


==Changelog==
<div style="height: 280px; overflow: auto; padding: 3px; border:1px solid #AAAAAA; font-size: small">
'''0.1.5028.0 : 2011.09.11'''
'''0.1.5028.0 : 2011.09.11'''
* Fixed shift of hashvalues (_Display_ only)
* Fixed shift of hashvalues (_Display_ only)
Line 206: Line 370:
* Fixed --Mon/--TOut to actually work
* Fixed --Mon/--TOut to actually work
* Fixed some bug in the old CLI converter
* Fixed some bug in the old CLI converter
* <more ponies>
* <no ponies>


'''0.1.5000.0 : 2011.09.11'''
'''0.1.5000.0 : 2011.09.11'''
Line 214: Line 378:
'''0.1.4964.0 : 2011.09.05'''
'''0.1.4964.0 : 2011.09.05'''
* all previous builds have been disabled
* all previous builds have been disabled
* Async ACreq sending
* Sending ACReqs asynchronous
* <insert ponies riding down the rainbow and other stuff that has happened>
* <insert ponies riding down the rainbow and other stuff that has happened>


staff
1,114

edits

MediaWiki spam blocked by CleanTalk.
MediaWiki spam blocked by CleanTalk.