Avdump2 on Linux

From AniDB
Revision as of 23:34, 27 January 2020 by Zeromind (talk | contribs) (→‎The Basics: add dotnet section)
Jump to navigation Jump to search

Avdump2 was developed on Windows, for Windows. Thanks to Mono it’s possible to run Avdump2 on Linux and all efforts are made to support all features on Linux as well, but handling every peculiarity of every Mono version on every Linux installation is beyond the scope of Avdump2 at this time. Therefore, some things just don’t work on Linux, or at least not right away, without some workarounds.

This page is a collection of various workarounds and suggestions that should help you get started with Avdump2 on Linux. If you need more assistance, try to catch CatWEax on IRC.

The Basics

Can't get Avdump2 to work at all? Here are some basic suggestions to set up Adump2.

Mono

Avdump2 has successfully been tested with Mono 2.4 on Ubuntu, with Mono 2.8 on Gentoo, and Mono 5.20 on openSUSE Tumbleweed.

Avdump2CL.exe looks like a Windows .exe, how do I start it?

Try making the file executable. If that doesn’t work, call it using the mono Interpreter: mono Avdump2CL.exe

OutOfMemory exceptions

On 32 bit systems Avdump2 sometimes dies with that error when processing many files at once due to heap memory fragmentation. If you use a donelog you can just restart Avdump2 and it’ll continue from where it crashed.

Mono complains about not being able to load type 'AVDump2CL.CL'

The actual error message probably looks similar to this:

Unhandled Exception: System.TypeLoadException: Could not load type 'AVDump2CL.CL' from assembly 'AVDump2CL, Version=0.1.6438.0, Culture=neutral, PublicKeyToken=null'.

If you get this error then you probably don’t have all necessary mono packages installed. On a Debian-based system make sure that you have the packages libmono-corlib2.0-cil and libmono-system2.0-cil installed.

libMediaInfo

Avdump2 only works with the libMediaInfo it comes with. If it can’t find that you will usually get MediaInfoLib is missing. It might start with a different version from elsewhere on your system, but then you’ll run into odd errors when you try to dump a file.

There are many possible causes if the library can’t be found.

Library search path missing

Most likely Mono can’t find the binary library even if it’s in the same directory as Avdump2CL.exe. Then you’ll have to set the environment variable LD_LIBRARY_PATH to point to the directory into which you extracted Avdump2. Write yourself a wrapper script to do that or have a look at the example wrapper scripts below. Sometimes you’ll also have to set MONO_PATH to point to the same directory.

Incompatible GLIBC

If your Linux installation is very old or you’re using a distribution that ships with an outdated GLIBC (CentOS is a prime offender here), Mono can’t link against libMediaInfo and Avdump2 will still complain about missing the library. To check if that’s the case, use ldd libMediaInfo*.so. If you get a message like version 'GLIBCXX_3.4.9' not found, you’re out of luck. You can either try to update GLIBC, which usually involves pain, or upgrade or change your distribution to one with a newer GLIBC.

Hashing works but I can’t dump anything

If you modified any of the required files Avdump2 came with, you will either get a version error or several timeouts and no meaningful error message. So no, you can’t edit the config file either.

Failed to create MediaInfoProvider

Failed to create MediaInfoProvider

Invalid value returned by mil. This error may indicate that your LANG or LC_ALL environment variables are not set to use UTF-8 encoding.

This error appears if you try to process a file with special characters in its name and your locale settings don’t match the file system encoding. Nowadays almost all file names are encoded with UTF-8 on Linux, so your locale should use that as well. If you work remotely on a server that sets LANG to “C” by default, it won’t work. Try setting LC_ALL=en_US.UTF-8, because that locale is installed almost everywhere by default. LC_ALL takes precedence over LANG. Have a look at the example script below to see how to make the setting automatically.

If you have a file with bogus characters in its name, even changing the locale won’t help. You have to rename the file to something sensible.

dotnet

Avdump2 also works on Linux via dotnet; it is about 30% faster than running it via mono, but appears to work with ASCII filenames only.

Create a AVDump2CL.runtimeconfig.json with the following content: 

{
    "runtimeOptions": {
        "configProperties": {
            "System.GC.Server": true,
            "System.GC.Concurrent": true,
            "System.Threading.ThreadPool.MinThreads": 6,
            "System.Threading.ThreadPool.MaxThreads": 24
        },
        "tfm": "netcoreapp3.1",
        "framework": {
            "name": "Microsoft.NETCore.App",
            "version": "3.1.1"                                                                                                                                                                                                                                                                         
        },
        "applyPatches": true,
        "rollForwardOnNoCandidateFx": 1
    }
}

Move the Linux lib to the DLL filename so dotnet finds it 

mv MediaInfo_x64.dll MediaInfo_x64.dll.windows
cp -a libMediaInfo_x64.so MediaInfo_x64.dll

Workarounds

The cursor caret is gone and output resumes at the far right edge of the terminal window when I abort Avdump2 with ^C or it exits with some error.

Avdump2 doesn’t reset the console properly when you abort it or when it runs into a Mono-specific error. You can restore the cursor with setterm -cursor on and a simple echo if you write a wrapper for that. See examples below.

Can’t write log files to relative paths

Donelogs and the other options that create or update text files assume relative paths to be relative to Avdump2’s home directory. If you installed Avdump2 outside your home directory you most likely don’t have write permission there and writing the log will fail. With the --UseCWD option you can make relative paths relative to the current working directory. Absolute paths are also an option to place the logs where you have write permissions.

Wrapper Script

To make using Avdump2 more comfortable, you can use or modify this wrapper script to handle the workarounds. Save it as /usr/local/bin/avdump2cl, for example. This example assumes that you extracted Avdump2 to /usr/local/lib/avdump2.

A simple wrapper to set the LD_LIBRARY_PATH, check the locale for UTF-8 and to fix the cursor after using ^C: 

#!/bin/bash

AVDUMP2_HOME=/usr/local/lib/avdump2

export LD_LIBRARY_PATH=${AVDUMP2_HOME}

# fix bug with ncurses 6.1+ 
# mono/issues/6752
export TERM=xterm

# If libMediaInfo still can't be found, uncomment this.
#export MONO_PATH=${AVDUMP2_HOME}

# Make sure that the locale uses UTF-8 encoding.
# Remove this if your file system actually uses something else.
shopt -s extglob
if [ -z "$LC_ALL" ]; then
  LC_ALL="$LANG"
fi
if [ "$LC_ALL" == "${LC_ALL%[Uu][Tt][Ff]?(-)8}" ]; then
  LC_ALL=en_US.UTF-8
fi
export LC_ALL
export LANG="$LC_ALL"

if ! mono ${AVDUMP2_HOME}/AVDump2CL.exe "$@" ; then
  echo
  setterm -cursor on
fi

Making Your Life Easier

When you made sure everything works fine, if you just want to easily hash files and generate ED2K links for adding new files, you could use this roughly-made Python 2 script. (best paired with Nautilus or Nemo actions)

Please note that this script relies on Zenity extension to display the dialog containing the generated ED2K links. If you get an error running this script, please make sure you have it installed. 

#!/usr/bin/python

import sys, os, subprocess

### EDIT THE FOLLOWING ACCORDING TO YOUR INFO
AVDUMP2CL_PATH = "/path/to/your/AVDump2CL.exe"
USERNAME = " "
API_KEY = " "
### 

command = "mono %s --Auth=%s:%s --PrintEd2kLink -q" % (AVDUMP2CL_PATH, USERNAME, API_KEY)
for arg in sys.argv[1:]:
    command += " \"%s\"" % arg

output = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
output = output.stdout.readlines()

ed2k = ""
for line in output:
    if line[0:4] == "ed2k":
        ed2k += line

os.system("zenity --info --text '%s'" % ed2k.rstrip())

Things That Just Don’t Work

Some things just don’t work on Linux and you’ll have to live with that. :P

Avdump2 corrupts the terminal or crashes with a DisplayBuffer error when I resize the terminal window.

The output routine used to draw the Avdump2 progress bars is not very compatible with Mono’s absolutely positioned terminal output. Enlarging the terminal window usually results in writing over previous lines and shrinking the terminal window usually results in crashes because Avdump2 tries to write to areas that don’t exist anymore.

Solutions: Don’t resize the terminal window while Avdump2 is running or use the -q option to disable the progress bars.

Inconsistent exit codes

Avdump2 exits with a return code of 0 on some errors, so don’t rely on the return code. It exits with a non-zero code on most serious errors, though.

Retrieved from "https://wiki.anidb.net/index.php?title=Avdump2_on_Linux&oldid=23616"