Avdump3

AVDump is a tool to extract meta information from media files while at the same time calculating multiple hashes. Based on that information reports can be generated in multiple forms. Of particular interest is the ability to send those reports back to AniDB and thereby quickly filling in missing metadata for new files.

Quickstart

Since you’re here you probably just want to send AVMF Packages (or Dumps) to AniDB which will add metadata to previously (or soon to be) added files. If that doesn’t mean anything to you, you probably want to start with https://wiki.anidb.net/Content:Files and https://wiki.anidb.net/Auto-creqing and https://wiki.anidb.net/Tutorial:How_to_Add_Files_for_Dummies

Anyway lets keep this short:

1. Login into AniDB with your account and visit https://anidb.net/user/setting, there goto the Account tab and set a password for “UDP API Key”
   
2. Make sure Net Core 3.1 or higher is installed
   
3. Download the latest Version of AVD3 with the link in the section below and extract it somewhere.
   
4. Start a terminal and navigate to AVDump3
   
5. Use the following arguments as the bare minimum
   

   Windows: AVDump3CL.exe --Auth=<YourUserName>:<YourUdpApiPassword> <APathToTheFiles>
   

   Linux: dotnet AVDump3CL.dll --Auth=<YourUserName>:<YourUdpApiPassword> <APathToTheFiles>

6. Optional: See sections below to improve usage experience

Download

DOWNLOAD (ZIP) (Build 8134) MD5 = 88264b3b4df756a3a2d4f79ad60dd622

What’s new compared to AVD2?

AVD3 is a complete rewrite of AVD2 which uses .NET Core instead of .NET Framework and treats Linux as a first class citizen. So it should run on Linux just as well as on Windows. To name a few other big differences:

• Can process multiple files in parallel
• Uses native code to speed up hashing significantly
• More efficient reading
• Can move/rename files based on scripts
• More hash algorithms
• Latest version of MediaInfoLib (MIL) is being used (currently 20.03)
• Support for .vtt subtitle files
• Support for 32bit has been dropped
• Support for MacOS is not yet available (To add support https://github.com/DvdKhl/AVDump3/blob/master/AVDump3NativeLib/src/AVD3MirrorBuffer.c needs to be implemented, help would be appreciated)

Supported formats

Most major file formats are supported to some degree. While hashes are created for all file types, stream details remain shakey for others (notably 'swf'). Because of that for some filetypes (underlined) only the hashes get auto-creqed.

• Video files: asf/wmv, avi, flv, m2ts, mk3d, mkv, mov, mp4, mpg/mpeg, ogm, ogv, qt, rm/rmvb, swf, ts, webm
• Subtitle files: ass, idx, js, lrc, mks, pjs, rt, smi, srt, ssa, sub, sup, tmp, tts, txt, vtt, xss
• Audio files: aac, ac3, dts, dtshd, flac, m4a, mka, mp3, ogg, ra, thd, wav, wma
• Archive files: 7z, ace, rar, zip
• Linker files: mkv, smil

Arguments

For more detailed information please run AVD3 with --Help!

Special Arguments

• FROMFILE

If the first argument is called FROMFILE and the next argument is a file. Every line of that file is interpreted as a single argument. A line which starts with // is interpreted as a comment and ignored. See https://wiki.anidb.net/Talk:Avdump3 for an example.

• PRINTARGS

Prints the effective arguments used into the terminal.

FileDiscovery

Parameter	Shorthand	Description	Usage	Default Value
--Recursive	-R	Recursively descent into Subdirectories	--Recursive	False
--ProcessedLogPath	--PLPath	Appends the full filepath to the specified path	--ProcessedLogPath=<FilePath1>[:<FilePath2>...]
--SkipLogPath	--SLPath	Filepaths contained in the specified file will not be processed	--SkipLogPath=<FilePath1>[:<FilePath2>...]
--DoneLogPath	--DLPath	Will set --SkipLogPath and --ProcessedLogPath to the specified filepath	--DoneLogPath=<Filepath>
--WithExtensions	--WExts	Only/Don't Process files with selected Extensions	--WithExtensions=[-]<Extension1>[,<Extension2>...]
--Concurrent	--Conc	Sets the maximal number of files which will be processed concurrently. First param (max) sets a global limit. (path,max) pairs sets limits per path.	--Concurrent=<max>[:<path1>,<max1>;<path2>,<max2>...]	1

Processing

Parameter	Shorthand	Description	Usage	Default Value
--ProducerMinReadLength		How much data in MiB the reader has to read each time at minimum	--ProducerMinReadLength	1
--ProducerMaxReadLength		How much data in MiB the reader is allowed to read each time at most	--ProducerMaxReadLength	8
--PrintAvailableSIMDs		Print available CPU SIMDs	--PrintAvailableSIMDs	False
--PauseBeforeExit	--PBExit	Pause console before exiting	--PauseBeforeExit	False
--BufferLength	--BLength	Circular buffer size for hashing	--BufferLength=<Size in MiB>	64
--Consumers	--Cons	Select consumers to use. Use without arguments to list available consumers	--Consumers=<ConsumerName1>[,<ConsumerName2>...]

FileMove

Parameter	Shorthand	Description	Usage	Default Value
--Test		Test FileMove Settings	--FileMove.Test	False
--LogPath		A line is written for each file that has been moved/renamed. (OldPath => NewPath)	--FileMove.LogPath=<FilePath>
--Mode		Determines how the Pattern Argument is going to be interpreted: Inline: Script is directly entered as the argument File: A path pointing to the script file Placeholder: See example for --Pattern CSharpScript: Script in C# DotNetAssembly: Net Core assembly to be loaded	--FileMove.Mode=<None|PlaceholderInline|PlaceholderFile|CSharpScriptInline|CSharpScriptFile|DotNetAssembly>	None
--Pattern		Available Placeholders ${Name}: FullName, FileName, FileExtension, FileNameWithoutExtension, DirectoryName, SuggestedExtension, Hash-<Name>-<2|4|8|10|16|32|32Hex|32Z|36|62|64>-<OC|UC|LC>	--FileMove.Pattern=${DirectoryName}\${FileNameWithoutExtension}${SuggestedExtension}	${DirectoryName}\${FileNameWithoutExtension}${FileExtension}
--DisableFileMove		Don't move the file even if the Pattern says so	--FileMove.DisableFileMove	False
--DisableFileRename		Don't rename the file even if the Pattern says so	--FileMove.DisableFileRename	False
--Replacements		Replace substrings in the returned filepath	--FileMove.Replacements=<Match1>=<Replacement1>[;<Match2>=<Replacement2>...]

Reporting

Parameter	Shorthand	Description	Usage	Default Value
--PrintHashes		Print calculated hashes in hexadecimal format to console	--PrintHashes	False
--PrintReports		Print generated reports to console	--PrintReports	False
--Reports		Select reports to use. Use without arguments to list available reports	--Reports
--ReportDirectory	--RDir	Reports will be saved to the specified directory	--ReportDirectory=<Directory>	<The directory AVD3 resides in>
--ReportFileName		Reports will be saved/appended to the specified filename The following placeholders ${Name} can be used: FileName, FileNameWithoutExtension, FileExtension, ReportName, ReportFileExtension	--ReportFileName=<FileName>	${FileName}.${ReportName}.${ReportFileExtension}
--ExtensionDifferencePath	--EDPath	Logs the filepath if the detected extension does not match the actual extension	--EDPath=extdiff.txt
--CRC32Error		Searches the filename for the calculated CRC32 hash. If not present or different a line with the caluclated hash and the full path of the file is appended to the specified path The regex pattern should contain the placeholder ${CRC32} which is replaced by the calculated hash prior matching. Consumer CRC32 will be force enabled!	--CRC32Error=<Filepath>:<RegexPattern>	(, (?i)${CRC32})

Diagnostics

Parameter	Shorthand	Description	Usage	Default Value
--SaveErrors		Errors occuring during program execution will be saved to disk	--SaveErrors	False
--SkipEnvironmentElement		Skip the environment element in error files	--SkipEnvironmentElement	False
--IncludePersonalData		Various places may include personal data. Currently this only affects error files, which will then include the full filepath	--IncludePersonalData	Fale
--ErrorDirectory		If --SaveErrors is specified the error files will be placed in the specified path	--ErrorDirectory=<DirectoryPath>	<The directory AVD3 resides in>
--NullStreamTest		Use Memory as the DataSource for HashSpeed testing. Overrides any FileDiscovery Settings!	--NullStreamTest=<StreamCount>:<StreamLength in MiB>:<ParallelStreamCount>	0:0:0

Display

Parameter	Shorthand	Description	Usage	Default Value
--HideBuffers		Hides buffer bars	--HideBuffers	False
--HideFileProgress		Hides file progress	--HideFileProgress	False
--HideTotalProgress		Hides total progress	--HideTotalProgress	False
--ShowDisplayJitter		Displays the time taken to calculate progression stats and drawing to console	--ShowDisplayJitter	False
--ForwardConsoleCursorOnly		The cursor position of the console will not be explicitly set. This option will disable most progress output	--ForwardConsoleCursorOnly	False

AniDBAvmf

Parameter	Shorthand	Description	Usage	Default Value
--LocalPort	--LPort	Local UDP port used for ACReqing	--LPort=<localport>
--ACreqErrorPath		A line is added to the specified file for every error which occurred during an Avmf Package transmission	--ACreqErrorPath=<FilePath>
--Authentication	--Auth	Enables ACReqing when valid credentials are provided Visit https://anidb.net/user/setting (Account Tab) to set the api key	--Auth=<username>:<api_key>
--HostEndPoint	--Host	Change endpoint of AniDB UDP API server AddressFamily: Can be 4=IPv4, 6=IPv6 or U=Unspecified	--Host=<hostname>:<hostport>[:<AddressFamily>]	api.anidb.info:9002:U
--Timeout	--TOut	Sets the retry count and the timeout before resending the dump	--TOut=<seconds>:<retries>	20:3

AniDBMisc

Parameter	Shorthand	Description	Usage	Default Value
--UploadErrors		Enables the automatic upload of program errors. Please be aware that if --IncludePersonalData is enabled, personal data is uploaded as well!	--UploadErrors	False
--Ed2kLogPath		Appends the ED2K-Link after a file has been processed into specified file separated by a line feed character (\n) Consumer ED2K will be force enabled!	--Ed2KLogPath=<path>

Development

Source code for the Core project can be found at GitHub: https://github.com/DvdKhl/AVDump3

The source code for the AniDB Module is not available to the public.