Avdump3: Difference between revisions

From AniDB
Jump to navigation Jump to search
No edit summary
Line 32: Line 32:
For more detailed information please run AVD3 with <code>--Help</code>!
For more detailed information please run AVD3 with <code>--Help</code>!


'''NameSpace''': FileDiscovery '''–Recursive, -R''' Recursively descent into Subdirectories
'''NameSpace''': FileDiscovery  


'''–ProcessedLogPath, –PLPath''' Appends the full filepath to the specified path
'''–Recursive, -R'''  


'''–SkipLogPath, –SLPath''' Filepaths contained in the specified file will not be processed
Recursively descent into Subdirectories


'''–DoneLogPath, –DLPath''' Will set –SkipLogPath and –ProcessedLogPath to the specified filepath
'''–ProcessedLogPath, –PLPath'''  


'''–WithExtensions, –WExts''' Only/Don’t Process files with selected Extensions
Appends the full filepath to the specified path


'''–Concurrent, –Conc''' Sets the maximal number of files which will be processed concurrently. First param (max) sets a global limit. (path,max) pairs sets limits per path.
'''–SkipLogPath, –SLPath'''  


'''NameSpace''': Processing '''–ProducerMinReadLength'''
Filepaths contained in the specified file will not be processed
 
'''–DoneLogPath, –DLPath'''
 
Will set –SkipLogPath and –ProcessedLogPath to the specified filepath
 
'''–WithExtensions, –WExts'''
 
Only/Don’t Process files with selected Extensions
 
'''–Concurrent, –Conc'''
 
Sets the maximal number of files which will be processed concurrently. First param (max) sets a global limit. (path,max) pairs sets limits per path.
 
'''NameSpace''': Processing  
 
'''–ProducerMinReadLength'''


'''–ProducerMaxReadLength'''
'''–ProducerMaxReadLength'''
Line 50: Line 66:
'''–PrintAvailableSIMDs'''
'''–PrintAvailableSIMDs'''


'''–PauseBeforeExit, –PBExit''' Pause console before exiting
'''–PauseBeforeExit, –PBExit'''  


'''–BufferLength, –BLength''' Circular buffer size for hashing
Pause console before exiting


'''–Consumers, –Cons''' Select consumers to use. Use without arguments to list available consumers
'''–BufferLength, –BLength'''
 
Circular buffer size for hashing
 
'''–Consumers, –Cons'''  
 
Select consumers to use. Use without arguments to list available consumers


'''NameSpace''': FileMove '''–Test'''
'''NameSpace''': FileMove '''–Test'''
Line 62: Line 84:
'''–Mode'''
'''–Mode'''


'''–Pattern''' Available Placeholders ${Name}: FullName, FileName, FileExtension, FileNameWithoutExtension, DirectoryName, SuggestedExtension, Hash-<Name>-&lt;2|4|8|10|16|32|32Hex|32Z|36|62|64&gt;-&lt;OC|UC|LC&gt;
'''–Pattern'''  
 
Available Placeholders ${Name}: FullName, FileName, FileExtension, FileNameWithoutExtension, DirectoryName, SuggestedExtension, Hash-<Name>-&lt;2|4|8|10|16|32|32Hex|32Z|36|62|64&gt;-&lt;OC|UC|LC&gt;


'''–DisableFileMove'''
'''–DisableFileMove'''
Line 70: Line 94:
'''–Replacements'''
'''–Replacements'''


'''NameSpace''': Reporting '''–PrintHashes''' Print calculated hashes in hexadecimal format to console
'''NameSpace''': Reporting  
 
'''–PrintHashes'''  
 
Print calculated hashes in hexadecimal format to console
 
'''–PrintReports'''
 
Print generated reports to console
 
'''–Reports'''
 
Select reports to use. Use without arguments to list available reports
 
'''–ReportDirectory, –RDir'''
 
Reports will be saved to the specified directory
 
'''–ReportFileName'''
 
Reports will be saved/appended to the specified filename The following placeholders ${Name} can be used: FileName, FileNameWithoutExtension, FileExtension, ReportName, ReportFileExtension
 
'''–ExtensionDifferencePath, –EDPath'''
 
Logs the filepath if the detected extension does not match the actual extension


'''–PrintReports''' Print generated reports to console
'''–CRC32Error'''  


'''–Reports''' Select reports to use. Use without arguments to list available reports
Searches the filename for the calculated CRC32 hash. If not present or different a line with the caluclated hash and the full path of the file is appended to the specified path The regex pattern should contain the placeholder ${CRC32} which is replaced by the calculated hash prior matching. Consumer CRC32 will be force enabled!


'''–ReportDirectory, –RDir''' Reports will be saved to the specified directory
'''NameSpace''': Diagnostics


'''–ReportFileName''' Reports will be saved/appended to the specified filename The following placeholders ${Name} can be used: FileName, FileNameWithoutExtension, FileExtension, ReportName, ReportFileExtension
'''–SaveErrors'''  


'''–ExtensionDifferencePath, –EDPath''' Logs the filepath if the detected extension does not match the actual extension
Errors occuring during program execution will be saved to disk


'''–CRC32Error''' Searches the filename for the calculated CRC32 hash. If not present or different a line with the caluclated hash and the full path of the file is appended to the specified path The regex pattern should contain the placeholder ${CRC32} which is replaced by the calculated hash prior matching. Consumer CRC32 will be force enabled!
'''–SkipEnvironmentElement'''  


'''NameSpace''': Diagnostics '''–SaveErrors''' Errors occuring during program execution will be saved to disk
Skip the environment element in error files


'''–SkipEnvironmentElement''' Skip the environment element in error files
'''–IncludePersonalData'''  


'''–IncludePersonalData''' Various places may include personal data. Currently this only affects error files, which will then include the full filepath
Various places may include personal data. Currently this only affects error files, which will then include the full filepath


'''–PrintDiscoveredFiles'''
'''–PrintDiscoveredFiles'''


'''–ErrorDirectory''' If –SaveErrors is specified the error files will be placed in the specified path
'''–ErrorDirectory'''  
 
If –SaveErrors is specified the error files will be placed in the specified path


'''–NullStreamTest''' Use Memory as the DataSource for HashSpeed testing. Overrides any FileDiscovery Settings!
'''–NullStreamTest'''  


'''NameSpace''': Display '''–HideBuffers''' Hides buffer bars
Use Memory as the DataSource for HashSpeed testing. Overrides any FileDiscovery Settings!


'''–HideFileProgress''' Hides file progress
'''NameSpace''': Display


'''–HideTotalProgress''' Hides total progress
'''–HideBuffers'''  


'''–ShowDisplayJitter''' Displays the time taken to calculate progression stats and drawing to console
Hides buffer bars


'''–ForwardConsoleCursorOnly''' The cursor position of the console will not be explicitly set. This option will disable most progress output
'''–HideFileProgress'''  


'''NameSpace''': AniDBAvmf '''–LocalPort, –LPort''' Local UDP port used for ACReqing
Hides file progress
 
'''–HideTotalProgress'''
 
Hides total progress
 
'''–ShowDisplayJitter'''
 
Displays the time taken to calculate progression stats and drawing to console
 
'''–ForwardConsoleCursorOnly'''
 
The cursor position of the console will not be explicitly set. This option will disable most progress output
 
'''NameSpace''': AniDBAvmf  
 
'''–LocalPort, –LPort'''  
 
Local UDP port used for ACReqing


'''–ACreqErrorPath'''
'''–ACreqErrorPath'''


'''–Authentication, –Auth''' Enables ACReqing when valid credentials are provided Visit https://anidb.net/user/setting (Account Tab) to set the api key
'''–Authentication, –Auth'''  
 
Enables ACReqing when valid credentials are provided Visit https://anidb.net/user/setting (Account Tab) to set the api key
 
'''–HostEndPoint, –Host'''
 
Change endpoint of AniDB UDP API server AddressFamily: Can be 4=IPv4, 6=IPv6 or U=Unspecified
 
'''–Timeout, –TOut'''


'''–HostEndPoint, –Host''' Change endpoint of AniDB UDP API server AddressFamily: Can be 4=IPv4, 6=IPv6 or U=Unspecified
Sets the retry count and the timeout before resending the dump


'''–Timeout, –TOut''' Sets the retry count and the timeout before resending the dump
'''NameSpace''': AniDBMisc '''–UploadErrors'''  


'''NameSpace''': AniDBMisc '''–UploadErrors''' Enables the automatic upload of program errors
Enables the automatic upload of program errors


'''–Ed2kLogPath'''
'''–Ed2kLogPath'''

Revision as of 22:19, 8 August 2020

AVDump is a tool to extract meta information from media files while at the same time calculating multiple hashes. Based on that information reports can be generated in multiple forms. Of particular interest is the ability to send those reports back to AniDB and thereby quickly filling in missing metadata for new files.

Quickstart

Since you’re here you probably just want to sent AVMF Packages (or Dumps) to AniDB which will add metadata to previously (or soon to be) added files. If that doesn’t mean anything to you, you probably want to start with https://wiki.anidb.net/Content:Files and https://wiki.anidb.net/Auto-creqing and https://wiki.anidb.net/Tutorial:How_to_Add_Files_for_Dummies! .

Anyway lets keep this short:
1. Login into AniDB with your account and visit https://anidb.net/user/setting, there goto the Account tab and set a password for “UDP API Key”
2. Make sure Net Core 3.1 or higher is installed
3. Download the latest Version of AVD3 with the link in the section below and extract it somewhere.
4. Start a terminal and navigate to AVDump3
5. Use the following arguments as the bare minimum
5.1. Windows: AVDump3CL.exe --Auth=<YourUserName>:<YourUdpApiPassword> <APathToTheFiles>
5.2. Linux: dotnet AVDump3CL.dll --Auth=<YourUserName>:<YourUdpApiPassword> <APathToTheFiles>

Download

http://static.anidb.net/client/avdump2/avdump3_8134.zip

What’s new compared to AVD2?

AVD3 is a complete rewrite of AVD2 which uses .NET Core instead of .NET Framework and treats Linux as a first class citizen. So it should run on Linux just as well as on Windows. To name a few other big differences:

  • Can process multiple files in parallel
  • Uses native code to speed up hashing significantly
  • More efficient reading
  • Can move/rename files based on scripts
  • More hash algorithms

Arguments

For more detailed information please run AVD3 with --Help!

NameSpace: FileDiscovery

–Recursive, -R

Recursively descent into Subdirectories

–ProcessedLogPath, –PLPath

Appends the full filepath to the specified path

–SkipLogPath, –SLPath

Filepaths contained in the specified file will not be processed

–DoneLogPath, –DLPath

Will set –SkipLogPath and –ProcessedLogPath to the specified filepath

–WithExtensions, –WExts

Only/Don’t Process files with selected Extensions

–Concurrent, –Conc

Sets the maximal number of files which will be processed concurrently. First param (max) sets a global limit. (path,max) pairs sets limits per path.

NameSpace: Processing

–ProducerMinReadLength

–ProducerMaxReadLength

–PrintAvailableSIMDs

–PauseBeforeExit, –PBExit

Pause console before exiting

–BufferLength, –BLength

Circular buffer size for hashing

–Consumers, –Cons

Select consumers to use. Use without arguments to list available consumers

NameSpace: FileMove –Test

–LogPath

–Mode

–Pattern

Available Placeholders ${Name}: FullName, FileName, FileExtension, FileNameWithoutExtension, DirectoryName, SuggestedExtension, Hash-<Name>-<2|4|8|10|16|32|32Hex|32Z|36|62|64>-<OC|UC|LC>

–DisableFileMove

–DisableFileRename

–Replacements

NameSpace: Reporting

–PrintHashes

Print calculated hashes in hexadecimal format to console

–PrintReports

Print generated reports to console

–Reports

Select reports to use. Use without arguments to list available reports

–ReportDirectory, –RDir

Reports will be saved to the specified directory

–ReportFileName

Reports will be saved/appended to the specified filename The following placeholders ${Name} can be used: FileName, FileNameWithoutExtension, FileExtension, ReportName, ReportFileExtension

–ExtensionDifferencePath, –EDPath

Logs the filepath if the detected extension does not match the actual extension

–CRC32Error

Searches the filename for the calculated CRC32 hash. If not present or different a line with the caluclated hash and the full path of the file is appended to the specified path The regex pattern should contain the placeholder ${CRC32} which is replaced by the calculated hash prior matching. Consumer CRC32 will be force enabled!

NameSpace: Diagnostics

–SaveErrors

Errors occuring during program execution will be saved to disk

–SkipEnvironmentElement

Skip the environment element in error files

–IncludePersonalData

Various places may include personal data. Currently this only affects error files, which will then include the full filepath

–PrintDiscoveredFiles

–ErrorDirectory

If –SaveErrors is specified the error files will be placed in the specified path

–NullStreamTest

Use Memory as the DataSource for HashSpeed testing. Overrides any FileDiscovery Settings!

NameSpace: Display

–HideBuffers

Hides buffer bars

–HideFileProgress

Hides file progress

–HideTotalProgress

Hides total progress

–ShowDisplayJitter

Displays the time taken to calculate progression stats and drawing to console

–ForwardConsoleCursorOnly

The cursor position of the console will not be explicitly set. This option will disable most progress output

NameSpace: AniDBAvmf

–LocalPort, –LPort

Local UDP port used for ACReqing

–ACreqErrorPath

–Authentication, –Auth

Enables ACReqing when valid credentials are provided Visit https://anidb.net/user/setting (Account Tab) to set the api key

–HostEndPoint, –Host

Change endpoint of AniDB UDP API server AddressFamily: Can be 4=IPv4, 6=IPv6 or U=Unspecified

–Timeout, –TOut

Sets the retry count and the timeout before resending the dump

NameSpace: AniDBMisc –UploadErrors

Enables the automatic upload of program errors

–Ed2kLogPath