Avdump3: Difference between revisions

From AniDB
Jump to navigation Jump to search
No edit summary
Line 1: Line 1:
AVDump is a tool to extract meta information from media files while at the same time calculating multiple hashes. Based on that information reports can be generated in multiple forms. Of particular interest is the ability to send those reports back to AniDB and thereby quickly filling in missing metadata for new files.
AVDump is a tool to extract meta information from media files while at the same time calculating multiple hashes.  
 
Based on that information reports can be generated in multiple forms.  
Of particular interest is the ability to send those reports back to AniDB and thereby quickly filling in missing metadata for new files.


===Supported formats===
===Supported formats===
Line 140: Line 141:
Searches the filename for the calculated CRC32 hash. If not present or different a line with the caluclated hash and the full path of the file is appended to the specified path.
Searches the filename for the calculated CRC32 hash. If not present or different a line with the caluclated hash and the full path of the file is appended to the specified path.
The regex pattern should contain the placeholder "<CRC32>" which is replaced by the calculated hash prior matching.
The regex pattern should contain the placeholder "<CRC32>" which is replaced by the calculated hash prior matching.
===Cross Platform===
===Windows===


====Windows Shortcut====
====Windows Shortcut====
Line 174: Line 179:
{{eyecatch|Note|'''The ''udp_api_key'' is not your user password!!! see [[Avdump#Auto-creqing|Auto-creqing]] .}}
{{eyecatch|Note|'''The ''udp_api_key'' is not your user password!!! see [[Avdump#Auto-creqing|Auto-creqing]] .}}


===Privacy===
====Sample Exception Report====
====Sample Exception Report====
     <AVD3CLException thrownOn="2020-05-24 15:54:42.5387">
     <AVD3CLException thrownOn="2020-05-24 15:54:42.5387">

Revision as of 19:05, 12 June 2020

AVDump is a tool to extract meta information from media files while at the same time calculating multiple hashes. Based on that information reports can be generated in multiple forms. Of particular interest is the ability to send those reports back to AniDB and thereby quickly filling in missing metadata for new files.

Supported formats

Most major file formats are supported to some degree. While hashes are created for all file types, stream details remain shakey for others (notably 'swf'). Because of that for some filetypes (underlined) only the hashes get auto-creqed.

  • Video files: asf/wmv, avi, flv, m2ts, mk3d, mkv, mov, mp4, mpg/mpeg, ogm, ogv, qt, rm/rmvb, swf, ts, webm
  • Subtitle files: ass, idx, js, lrc, mks, pjs, rt, smi, srt, ssa, sub, sup, tmp, tts, txt, xss
  • Audio files: aac, ac3, dts, dtshd, flac, m4a, mka, mp3, ogg, ra, thd, wav, wma
  • Archive files: 7z, ace, rar, zip
  • Linker files: mkv, smil

Download

DOWNLOAD(ZIP) (Build xxxx) MD5 = xxxxxxxxxxxxxxxxxxxxxxxx

Quickstart

Avdump3 is a simple command line client:

AVDump3CL.exe [--<Arg>[=<Value>] -<OneLetterArg> ...] <media file/folder> [<media file/folder> ...]

Arguments

NameSpace: AniDB

--UploadErrors | (False) Enables the automatic upload of program errors.

--Authentication, --Auth | --Auth=<username>:<api_key> (<none given>) Enables ACReqing when valid credentials are provided. Also enables error Reporting! (Add --NoEUp to disable) Visit anidb.net/perl-bin/animedb.pl?show=profile (Account Tab) to set the api key

--HostEndPoint, --Host | --Host=<hostname>:<hostport> (api.anidb.info:9002:U) Change endpoint of AniDB UDP API server

--LocalPort, --LPort | --LPort=<localport> (0) Local UDP port used for ACReqing

--Timeout, --TOut | --TOut=<seconds>:<retries> (20:3) Sets the retry count and the timeout before resending the dump

--ACreqErrorPath | (<null>) --Ed2KLogPath | --Ed2KLogPath=<path> (<null>) Appends the ED2K-Link after a file has been processed into specified file separated by a line feed character (\n)


NameSpace: Diagnostics

--SaveErrors | --SaveErrors (False) Errors occuring during program execution will be saved to disk.

--SkipEnvironmentElement | --SkipEnvironmentElement (False) Skip the environment element in error files

--IncludePersonalData | --IncludePersonalData (False) Various places may include personal data. Currently this only affects error files, which will then include the full filepath

--ErrorDirectory | If --SaveErrors is specified the error files will be placed in the specified path (D:\Projects\C#\AVDump3\AVDump3CL\bin\x64\Debug\netcoreapp3.1) --ErrorDirectory=<DirectoryPath>

--NullStreamTest | --NullStreamTest=<StreamCount>:<StreamLength in MiB>:<ParallelStreamCount> () Use Memory as the DataSource for HashSpeed testing. Overrides any FileDiscovery Settings!


NameSpace: Display

--HideBuffers | --HideBuffers (False) Hides buffer bars

--HideFileProgress | --HideFileProgress (False) Hides file progress

--HideTotalProgress | --HideTotalProgress (False) Hides total progress

--ShowDisplayJitter | --ShowDisplayJitter (False) Displays the time taken to calculate progression stats and drawing to console

--ForwardConsoleCursorOnly | --ForwardConsoleCursorOnly (False) The cursor position of the console will not be explicitly set. This option will disable most progress output.


NameSpace: FileDiscovery

--Recursive, -R | --Recursive (False) Recursively descent into Subdirectories

--ProcessedLogPath, --PLPath | --ProcessedLogPath=<Filepath> (<null>) Appends the full filepath to the specified path

--SkipLogPath, --SLPath | --SkipLogPath=<FilePath> (<null>) Filepaths contained in the specified file will not be processed

--DoneLogPath, --DLPath | --DoneLogPath=<Filepath> (<null>) Will set --SkipLogPath and --ProcessedLogPath to the specified filepath

--Concurrent, --Conc | --Concurrent=<max>[:<path1>,<max1>;<path2>,<max2>;...] (1) Sets the maximal number of files which will be processed concurrently. First param (max) sets a global limit. (path,max) pairs sets limits per path.

--WithExtensions, --WExts | --WithExtensions=[-]<Extension1>[,<Extension2>,...] () Only/Don't Process files with selected Extensions


NameSpace: Processing

--BufferLength, --BLength | --BufferLength=<Size in MiB> (64) Circular buffer size for hashing

--ProducerMinReadLength | (1) --ProducerMaxReadLength | (8) --Consumers, --Cons | --Consumers=<ConsumerName1>[,<ConsumerName2>,...] () Select consumers to use. Use without arguments to list available consumers

--PrintAvailableSIMDs | (False) --PauseBeforeExit, --PBExit | --PauseBeforeExit (False) Pause console before exiting


NameSpace: Reporting

--PrintHashes | --PrintHashes (False) Print calculated hashes in hexadecimal format to console

--PrintReports | --PrintReports (False) Print generated reports to console

--Reports | --Reports (<null>) Select reports to use. Use without arguments to list available reports

--ReportDirectory, --RDir | --ReportDirectory=<Directory> (D:\Projects\C#\AVDump3\AVDump3CL\bin\x64\Debug\netcoreapp3.1) Reports will be saved to the specified directory

--ReportFileName | --ReportFileName=<FileName> (<FileName>.<ReportName>.<ReportFileExtension>) Reports will be saved/appended to the specified filename. The following placeholders can be used: <FileName>, <FileNameWithoutExtension>, <FileExtension>, <ReportName>, ReportFileExtension

--ExtensionDifferencePath, --EDPath | --EDPath=extdiff.txt () Logs the filepath if the detected extension does not match the actual extension

--CRC32Error | --CRC32Error=<Filepath>:<RegexPattern> ((, (?i)<CRC32>)) Searches the filename for the calculated CRC32 hash. If not present or different a line with the caluclated hash and the full path of the file is appended to the specified path. The regex pattern should contain the placeholder "<CRC32>" which is replaced by the calculated hash prior matching.

Cross Platform

Windows

Windows Shortcut

  1. Extract Avdump to a folder of choice.
    Example: "C:\Program Files\AVDump2\"
  2. Right-click AVDump2CL.exe, select Create Shortcut.
  3. Right-click the shortcut, add options to the Shortcut - Target line.
    Example, normal usage: "C:\Program Files\AVDump2\AVDump2CL.exe" -p
    Example, auto-creq usage: "C:\Program Files\AVDump2\AVDump2CL.exe" --Auth=myName:udp_api_key
    You can also set other options from the list above if you wish to do so.
    A commandline option to export ED2K links to a file is by adding --Exp=C:\Path\ed2k.txt, etc.
    Note that you need to set an UDP API Key in your AniDB profile for auto-creqing.
    Optional: Change Run to Maximized.
  4. Drag and drop any media-file onto the shortcut.
Note The udp_api_key is not your user password!!! see Auto-creqing .

Send to

  1. Move the shortcut created above to the SendTo folder.
    Example, under XP: "C:\Documents and Settings\Username\SendTo"
    Example, under Vista: "C:\Users\Username\AppData\Roaming\Microsoft\Windows\SendTo"
    Where Username is your Windows Login-Name.
    Note that SendTo is a hidden folder.
  2. Right-click any media-file, go to "Send To", choose AVDump2CL.exe
  3. Get CReqing

MS-DOS Batch File

  1. Extract Avdump to a folder of choice.
  2. Create a MS-DOS batch file with the 2 lines below, and save it as filename.bat.
  3. @ECHO OFF
  4. C:\Program Files\AVDump\AVDump2CL.exe %* --Exp=C:\Path\ed2k.txt --Auth=myName:udp_api_key
  5. Drag and drop any media-file onto the shortcut.
Note The udp_api_key is not your user password!!! see Auto-creqing .

Privacy

Sample Exception Report

   <AVD3CLException thrownOn="2020-05-24 15:54:42.5387">
       <Information>
           <EntryAssemblyVersion>3.0.8067.0</EntryAssemblyVersion>
           <LibVersion>3.0.8067.0</LibVersion>
           <Session>177c9119-7ac3-4c78-96cb-680f83ee5c48</Session>
           <Framework>3.1.4</Framework>
           <OSVersion>Microsoft Windows NT 6.2.9200.0</OSVersion>
           <IntPtr.Size>8</IntPtr.Size>
           <Is64BitOperatingSystem>true</Is64BitOperatingSystem>
           <Is64BitProcess>true</Is64BitProcess>
           <ProcessorCount>8</ProcessorCount>
           <UserInteractive>true</UserInteractive>
           <SystemPageSize>4096</SystemPageSize>
           <WorkingSet>254959616</WorkingSet>
       </Information>
       <Message>Unhandled AppDomain wide Exception</Message>
       
       <Cause>
           <ArgumentException>
               <Message>Waithandle array may not be empty. (Parameter 'waitHandles')</Message>
               <Stacktrace>
                   <Frame>at System.Threading.WaitHandle.WaitMultiple(ReadOnlySpan`1 waitHandles, Boolean waitAll, Int32 millisecondsTimeout)</Frame>
                   <Frame>at System.Threading.WaitHandle.WaitMultiple(WaitHandle[] waitHandles, Boolean waitAll, Int32 millisecondsTimeout)</Frame>
                   <Frame>at System.Threading.WaitHandle.WaitAll(WaitHandle[] waitHandles)</Frame>
                   <Frame>at AVDump3CL.AVD3CLModule.Process(String[] paths) in D:\Projects\C#\AVDump3\AVDump3CL\AVD3CLModule.cs:line 261</Frame>
                   <Frame>at AVDump3CL.Program.Main(String[] args) in D:\Projects\C#\AVDump3\AVDump3CL\Program.cs:line 101</Frame>
               </Stacktrace>
               
           </ArgumentException>
       </Cause>
       <Stacktrace />
   </AVD3CLException>