UDP API Definition

From AniDB
Revision as of 16:20, 25 October 2005 by Epoximator (talk | contribs) (conv udp spec to wiki, plan is to dev v3)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to navigation Jump to search

Author: Exp
Version: 0.02c - 10.10.2004
Version number used for protover parameter: "2"

Updated versions of this document, notifications and official clients:

If you want to recieve a notification mail every time a new version of the API Specs is released or want to have your client linked/hosted by anidb drop Exp a line.

IMPORTANT: If you are writing a client please post your client name and your email address here RIGHT NOW. Thanks!


Changelog

 0.02c - 10.10.2004
 LOGOUT now requires the session string too
 Note on 6xx server errors and tags added
 
 0.02b - 14.08.2004
 503 INVALID SESSION changed to 506 INVALID SESSION
 BUGFIX: 200 message on AUTH did not contain a session key
 BUGFIX: 1 internal bug fixed
 
 0.02 - 13.08.2004
 new session key needs to be send with each command
 random udp ports no longer required
 FILE, MYLIST, MYLISTDEL, MYLISTADD commands added
 Some textparts of return values changed
 additional documentation added
 check the caching section!
 
 0.01b - 12.08.2004
 added NOTIFYLIST, NOTIFYGET, NOTIFYACK, SENDMSG commands
 
 0.01 - 10.08.2004
 initial version


Formats used in this Spec

  • {vartype varname} - Specifies what shall be inserted at this point and its type
  • [...] - Everything between [ and ] is optional

Used types

  • int2 - 2byte Integer (in string representation)
  • int4 - 4byte Integer (in string representation)
  • str - String (NOTE: might be as long as 5000 bytes !!!, via udp api no string is likely to extend >1500 bytes)

Content Encoding

  • Current character encoding used by the api: unknown
  • Escape scheme for option values: html form encoding + newline
this means you have to encode at least & and = in your option values in html form encoding style before sending them to the api server. All other html form encodings are optional.
  • All newlines should be replaced by <br>
escape scheme for returned data fields: ', | and newline
newlines are encoded as <br>, | is not allowed in data fields and ' is encoded as ´.


Basics

IDs

All IDs start at 1 (not 0). A normal table index (id) may never be 0 (there are some exceptions where an entry with id=0 exists, however those represent unknown values) It is possible for table entries to have id fields (i.e. gid) with a value of 0. Those are to be interpreted as "NONE" or "NULL". An ID is NEVER reused. That means after an entry is deleted no new entry will ever have that ID again.

IMPORTANT: A client should ignore any additional trailing data fields it doesn't expect. Additional fields are going to be added to the output of some commands from time to time. Those additional fields will simply be appended to the end of the existing output.

A client should be written in a way so that it is not affected by such new fields.

NOTE: Due to the length contraints of an UDP package API replies might be truncated at the end if they exceed the max. length of ~1500 bytes. Such events are not handled specially, the data will simply be send truncated.


Server / UDP Connection:

The Client has to send data packets to:

Server: api.anidb.info
Port: 9000/UDP


IMPORTANT: Maintenance

While in daily maintenance the AniDB API will reply with

601 ANIDB OUT OF SERVICE - TRY AGAIN LATER

to all commands.

If a client recieves such a return value it should wait at least 30 minutes before trying again.

NOTE: 6xx messages do not always return the tags given with the command which caused the error!


Connection Problems

There are many ways for a client to end up banned or the API might currently be handling too many concurrent connections. If a client does not get any reply to an AUTH command it should start to increase the retry delay exponentially with every failed login attempt. (i.e. try again after 30 seconds if the first login attempt failed, if the second fails too retry after 2 minutes, then 5 minutes, then 10 minutes, then 30 minutes, ... until you reach a retry delay of ~2-4h.)


Local Port

A client should select a fixed local port >1024 at install time and reuse it for local UDP Sockets. If the API sees too many different UDP Ports from one IP within ~1 hour it will ban the IP. (So make sure you're reusing your UDP ports also for testing/debuging!)

The local port may be hardcoded, however, an option to manually specify another port should be offered.

The servername and port should not be hardcoded into a frontend but should be read from a configuration file. The network communication is packet and line based. Each ANIDB API command is exactly one UDP packet containing one line. Results are send as one packet but may consist out of multiple lines. A return value always starts with a 3 byte result code followed by a human redable version of the result code. Be aware that important data fields may be returned directly after the return code (see: 230 code). The meaning for all result codes can be found in this document. If there is more than one entry returned, it's one entry per line. Lines are terminated by a \n (no dos linefeed \r). The elements of a format string are seperated by a "|" character.


Flood Protection

To prevent high server load the UDP API server enforces a strict flood protection policy.

  • Short Term:
A Client MUST NOT send more than 0.5 packets per second.
The server will start to enforce the limit after the first 5 packets have been recieved.
  • Long Term:
A Client MUST NOT send more than one packet every 30 seconds over an extended amount of time.

Once a client exceeds a rate limit all further UDP packets from that client will be dropped without feedback until the client's packet rate is back down to acceptable levels.

NOTE: Dropped packets are still taken into account for the packet rate. Meaning if you continuously send packets your client will be banned forever.

Generally clients should be written in a way to minimize server and network load. You should always keep that in Mind. Abusive clients may be banned completly.

NOTE: If you suddenly stop getting replies from the AniDB API or you normally don't have a noteable packetloss and from one point on you suddenly note a high packetloss percentage you have most likely entered a critical flood ratio.

If the AniDB API isn't answering at all it might either be down or you have been banned, this is normally caused by vialoating the API specs (i.e. too many connections using different ports from one ip, too many auth failures, ...) (usually lasts 30 minutes). If you're experiencing packet loss you have probably reached the rate limit and the API starts to randomly drop incoming packets from your IP. (can be solved by enforcing a delay between multiple commands when you're sending a batch)


Tag option

The api will add a user defined string at the beginning of each reply line seperated with a space, if desired. To enable this you have to add the tag={str usertag} option to a command. A tag is only valid for the command it was send with, meaning it is not persistent, if you want to have tags in front of all reply lines you will have to append the tag option to each command you send to the server. Tags are ment to enable a client to handle more than one request/reply at a time.

Usage example: 

 AUTH user=baka&pass=baka&protover=25&client=someclient&clientver=1&tag=abc123
 abc123 200 Jxqxb LOGIN ACCEPTED

or 

 LOGOUT s=Jxqxb&tag=byebye
 byeybe 203 LOGGED OUT
NOTE: The tag option is valid for all api commands and is thus not explicitly listed in the description of each command.


Caching

To minimize server and network load a client should use local caching in order to prevent repeated API requests for the same data.

A client should locally cache FILE/EP/ANIME/GROUP/... info whereever possible for extended amounts of time. (I.e. if a client is used to scan a local folder with anime files and add them via API to a users mylist then it shall only ask for all files in the first run and cache the info for all files known to AniDB. If run again over the same folder it shall only check those files which were unknown to anidb at the time of the last check.)

Later versions of the API might enforce this by banning clients which ask for the same information more than once every week/month.


Server Errors

At any time the API might return a fatal error of the form:

  • 6xx ERROR DESCRIPTION

Possible codes are 600-699.

Occurances of 6xx errors (other than 601) should be reported to me.

NOTE: 6xx messages do not always return the tags given with the command which caused the error!


Commands

NOTE: _ANY_ command which requires parameters may return: 505 ILLEGAL INPUT OR ACCESS DENIED

AUTH: Authing to the AnimeDB

Command String:

  • AUTH user={str username}&pass={str password}&protover={int4 apiversion}&client={str clientname}&clientver={int4 clientversion}

Possible Replies:

  • 200 {str session_key} LOGIN ACCEPTED
  • 201 {str session_key} LOGIN ACCEPTED - NEW VERSION AVAILABLE
  • 500 LOGIN FAILED
  • 503 CLIENT VERSION OUTDATED
  • 504 CLIENT BANNED - {str reason}
  • 505 ILLEGAL INPUT OR ACCESS DENIED
  • 601 ANIDB OUT OF SERVICE - TRY AGAIN LATER

IMPORTANT! POSSIBLE RETURN CODE AT ANY POINT OF THE API PROTOCOL:

  • 501 LOGIN FIRST
  • 502 ACCESS DENIED
  • 506 INVALID SESSION

Info:

  • The session_key is a String containing only a-z,A-Z,0-9 chars of a length of 4-8 characters.
It has to be stored by the client and needs to be send as parameter with every command which requires the user to logged in.
  • The session_key String is appended as parameter "s", i.e. "PUSH notify=1&msg=1&s={str session_key}".
  • On a 500 LOGIN FAILED message the client should request the user to enter username and password again.
  • In case of a 501 LOGIN FIRST message the client should silently resend an auth command and send the failed command again.
  • A 502 ACCESS DENIED message should abort the current action on the client side and display a message to the user.
  • A 506 INVALID SESSION means that either the session key parameter "s" was not provided with a command that requires it or the session key is no longer valid. The client should reissue an AUTH command.
NOTE: A frontend shall expect 501 and 502 messages to be returned on ANY command.
NOTE: Anidb usernames are always lowercase and may only contain characters (a-z) and numbers (0-9).
  • The client should silently convert all entered usernames to lowercase before sending them to the API.
  • The client should send the apiversion of the AnimeDB API version it supports as value of the protover parameter.
  • The client MAY NOT send anything but the version of the API Specs the author used to write the client! (as it is stated at the top of this file @ "Version number used for protover parameter")
The API will compare that value to it's own version of the API and if the version of the client is older the API will decide wheter the changes were significant enough to deny the old client access to the DB.
  • The clientname shall be a lowercase string containing only the chars a-z of 4-16 chars length which identifies the client. (i.e. mykickassclient)
  • The clientversion shall be a number starting with 1, increased on every change in the client.
clientname and clientversion might be used by the API to distinguish between different clients and client versions if that should ever become nessecary.
IMPORTANT:
  • DO NOT use the clientname of another existing client.
  • ALWAYS increase the clientversion number if you release a new client version.
  • A Login and its assigned session_key is valid until the virtual UDP connection times out or until a LOGOUT command is issued.
  • The virtual UDP connection times out if no data was recieved from the client for 35 minutes.
  • A client should issue a NOTIFY command once every 30 minutes to keep the connection alive should that be required.
  • If the client does not use any of the notification/push features of the API it should NOT keep the connection alive, furthermore it should explicitly terminate the connection by issueing a LOGOUT command once it finished it's work.
  • If it is very likely that another command will be issued shortly (within the next 20 minutes) a client may keep the current connection open, until it times out on it's own, by not sending a LOGOUT command.
  • The client shall notify the user if it recieved a 201 message at login.
This means a new version of the client is available, however the old version is still supported otherwise a client banned message would have been returned.
  • The user should get a popup message on 503 and 504 messages telling him to update his client software.
(NOTE: 504 means that this version of the client is banned, not the user!)
IMPORTANT: Make sure your client handels ALL possible AUTH return codes before giving out any versions!

LOGOUT: Logout

Command String:

  • LOGOUT

Possible Replies:

  • 203 LOGGED OUT
  • 403 NOT LOGGED IN

Info:

  • This command only works if you are already logged in.
  • A logout should ALWAYS be issued if the client is currently logged in and is either exiting or not expecting to send/recieve any anidb api packets for the next >= 30 minutes.

PUSH: UDP Notification Registration

Command String:

  • PUSH notify={bool push_file_added_notifications}&msg={bool push_msg_added_notifications}

Possible Replies:

  • 270 NOTIFICATION ENABLED
OR (if both values are 0)
  • 370 NOTIFICATION DISABLED

Note:

  • This command only works if you are logged in.
  • With this command you can register your client as an observer for anidb notification and message events for the current user. If you are registered for one or both events the anidb server will send an UDP packet (format see below) on each change which affects the current user. The UDP packet is sent to the ip and port from which the AUTH command was recieved.
  • UDP packets are resend 3 times, if the client does not acknowlege them. After that the client is logged out and no further notifies will be send.
  • On recieving any of the above packets the client has to issue a PUSHACK command with the notify_packet_id provided in the notification packet.

Notification Packet Format:

  • File Added Notify:
 271 {int4 notify_packet_id} NOTIFICATION
 {int4 aid}|{int4 date}|{int4 count}|{str animetitle}
aid is the id of the affected anime
date is the time of the event (in seconds since 1.1.1970)
count is the number of events pending for this anime
animetitle is the name of the affected anime
  • Message Added Notify:
 272 {int4 notify_packet_id} NOTIFICATION
 {int2 type}|{int4 date}|{int4 senderuid}|{str sendername}|{str subject}
type is the type of the message (0=normal msg, 1=annonymous, 2=system msg, 3=mod msg)
date is the time the message was sent (in seconds since 1.1.1970)
senderuid/sendername are the user id and username of the sender
subject is the message subject

PUSHACK: UDP Notification Acknowledge

Command String:

  • PUSHACK nid={int4 notify_packet_id}

Possible Replies:

  • 280 PUSHACK CONFIRMED
  • 380 NO SUCH PACKET PENDING

Info:

  • This command only works if you are logged in.

see: PUSH


NOTIFY: Notifications

Command String:

  • NOTIFY

Possible Replies:

  • 290 NOTIFICATION
{int4 pending_notifies}|{int4 pending_msgs}
  • 501 LOGIN FIRST

Info:

  • This command only works if you are logged in.
  • pending_notifies number of animes which have pending notifications for this user.
  • pending_msgs number of unread anidb messages for this user.
  • A client which has registered to recieve UDP notification packets should issue this command once every 30 minutes to keep the connection alive and to ensure that it has not been logged out by the API server yet.
  • If the client did send a NOTIFY within the last 35 minutes and it was confirmed by AniDB then recieving a 501 LOGIN FIRST message for the next NOTIFY command shows that AniDB logged the client out bc it did not respond to a PUSH Notification packet.
  • There is no command to retrieve missed PUSH Notifications.
NOTE: This command MUST NOT be issued more than once every 20 minutes.


NOTIFYLIST: List Notification/Message IDs

Command String:

  • NOTIFYLIST

Possible Replies:

  • 291 NOTIFYLIST
{str type}|{int4 id}
{str type}|{int4 id}
...
  • 501 LOGIN FIRST

Info:

  • This command only works if you are logged in.
  • type is:
M for message entries
N for notification entries
  • id is the identifier for the notification/message as required by NOTIFYGET.
  • NOTIFYLIST returns one line per entry, if no entries are available only the first line of the reply is returned.
NOTE: This command MUST NOT be issued regulary but should only be triggered by either a push message recieved by the client or a reply to a NOTIFY command which tells you that there are messages/notifications waiting.


NOTIFYGET: Get Notification/Message

Command String:

  • NOTIFYGET type={str type}&id={int4 id}

Possible Replies:

  • 292 NOTIFYGET
[if type = M]
{int4 id}|{int4 from_user_id}|{str from_user_name}|{int4 date}|{int4 type}|{str title}|{str body}
[if type = N]
{int4 aid}|{int4 type}|{str count}|{int4 date}|{str anime_name}
  • 393 NO SUCH ENTRY
  • 505 ILLEGAL INPUT OR ACCESS DENIED
  • 501 LOGIN FIRST

Info:

  • This command only works if you are logged in.
  • type is:
M for message entries
N for notification entries
  • id is the identifier for the notification/message as required by NOTIFYGET.
  • for message entries (M):
date is the time of the event (in seconds since 1.1.1970)
type is the type of the message (0=normal msg, 1=annonymous, 2=system msg, 3=mod msg)
  • for notification entries (N):
aid is the id of the affected anime
type is the notification type (0=all, 1=new, 2=group)
count is the number of events pending for this anime
date is the time of the event (in seconds since 1.1.1970)


NOTIFYACK: Ack Notification/Message

Command String:

  • NOTIFYACK type={str type}&id={int4 id}

Possible Replies:

  • 282 NOTIFYACK SUCCESSFUL
  • 382 NO SUCH ENTRY
  • 505 ILLEGAL INPUT OR ACCESS DENIED
  • 501 LOGIN FIRST

Info:

  • This command only works if you are logged in.
  • This command will mark a message read or clear a notification.
  • type is:
M for message entries
N for notification entries

SENDMSG: Send Message

Command String:

  • SENDMSG to={str username}&title={str title}&body={str body}

Possible Replies:

  • 294 SENDMSG SUCCESSFUL
  • 394 NO SUCH USER
  • 501 LOGIN FIRST

Note:

  • This command only works if you are logged in.
  • This command allows you to send an AniDB message.
IMPORTANT: title must not be longer than 100 chars.

body must not be longer than 1200 chars.

FILE: Retrieve File Data

Command String:

query by fid:
  • FILE fid={int4 id}
query by size+ed2k hash:
  • FILE size={int4 size}&ed2k={str ed2khash}

Possible Replies:

  • 220 FILE
{int4 fid}|{int4 aid}|{int4 eid}|{int4 gid}|{int4 state}|{int4 size}|{str ed2k}|{str anidbfilename}
  • 320 NO SUCH FILE

Info:

  • fid, aid, eid, gid are the unique ids for the file, anime, ep, group entries at anidb.
You can use those to create links to the corresponding pages at anidb.
  • anidbfilename is the anidb filename for the file.
However this name does not contain all the extra information of the filenames on AniDB and might be composed slightly different.
  • file state:
 bit / int value 	meaning 	
 1 / 1 		FILE_CRCOK: file matched official crc (displayed with green background in anidb)	
 2 / 2 		FILE_CRCERR: file DID NOT match official crc (displayed with red background in anidb)	
 3 / 4 		FILE_ISV2: file is version 2	
 4 / 8 		FILE_ISV3: file is version 3	
 5 / 16 		FILE_ISV4: file is version 4	
 6 / 32 		FILE_ISV5: file is version 5
  • examples:
 state ==== 9 ==> FILE_CRCOK+FILE_ISV3 ==> file matched official crc and is version 3
 state ==== 0 ==> - ==> file was not crc checked and is version 1
 state ==== 34 ==> FILE_CRCERR+FILE_ISV5 ==> file DID NOT match official crc and is version 5
 state ==== 1 ==> FILE_CRCOK ==> file matched official crc and is version 1
 state ==== 8 ==> FILE_ISV3 ==> file was not crc checked and is version 3

MYLIST: Retrieve Mylist Data

Command String:

by lid: (mylist id)
  • MYLIST lid={int4 lid}
by fid:
  • MYLIST fid={int4 fid}
by size+ed2k hash:
  • MYLIST size={int4 size}&ed2k={str ed2khash}

Possible Replies:

  • 221 MYLIST
{int4 lid}|{int4 fid}|{int4 eid}|{int4 aid}|{int4 gid}|{int4 date}|{int2 state}|{int4 viewdate}|{str storage}|{str source}|{str other}
  • 321 NO SUCH ENTRY

Info:

  • The state field provides information about the location and sharing state of a file in mylist.
  • If files are added after hashing, a client should specify the state as 1 (on hdd) (if the user doesn't explicitly select something else).
  • states:
 0 - unknown - state is unknown or the user doesn't want to provide this information
 1 - on hdd - the file is stored on hdd (but is not shared)
 2 - on cd - the file is stored on cd
 3 - deleted - the file has been deleted or is not available for other reasons (i.e. reencoded)
 4 - shared - the file is stored on hdd and shared
 5 - release - the file is stored on hdd and shared on release priority


Data Storage

MYLIST ADD: Add file to mylist

Command String:

by lid: (mylist id, edit only)
  • MYLISTADD lid={int4 lid}&edit=1[&state={int2 state}&viewed={boolean viewed}&source={str source}&storage={str storage}&other={str other}]
by fid:
  • MYLISTADD fid={int4 fid}[&state={int2 state}&viewed={boolean viewed}&source={str source}&storage={str storage}&other={str other}][&edit=1]
by size+ed2k hash:
  • MYLISTADD size={int4 size}&ed2k={str ed2khash}[&state={int2 state}&viewed={boolean viewed}&source={str source}&storage={str storage}&other={str other}][&edit=1]

Possible Replies:

  • 210 MYLIST ENTRY ADDED
  • 310 FILE ALREADY IN MYLIST
  • 311 MYLIST ENTRY EDITED
  • 410 NO SUCH FILE
  • 411 NO SUCH MYLIST ENTRY
  • 505 ILLEGAL INPUT OR ACCESS DENIED

Info:

  • All data except lid/fid/size+hash is optional.
  • Viewed should be 0 for unwatched files and 1 for watched files.
  • Other is the only field which may contrain newlines, but they have to be stored as
  • If edit is 1 then an existing entry, if found, will be updated with the given values.
  • For state values refer to: MYLIST
  • If you want to change an existing entry and already know it's mylist id (lid) please use the lid-version of this command. It put less load on the server.
NOTE: Currently this command does not update the internal caches of AniDB. Using this command will lead to wrong stats values/counts being displayed in your Mylist & co @ AniDB for upto 24h.
NOTE: {{{2}}}

MYLIST DEL: Remove file from mylist

'Command String:'

by lid: (mylist id)
  • MYLISTDEL lid={int4 lid}
by fid:
  • MYLISTDEL fid={int4 fid}

Possible Replies:

  • 211 ENTRY DELETED
  • 411 NO SUCH ENTRY
  • 505 ILLEGAL INPUT OR ACCESS DENIED

Info:

  • Currently this command does not update the internal caches of AniDB. Using this command will lead to wrong stats values/counts being displayed in your Mylist & co @ AniDB for upto 24h.


Misc Commands

PING: Ping Command

Command String:

  • PING

Possible Replies:

  • 300 PONG

Info:

  • May be used by a frontend to determin if the API is sill available.
  • May be executed even if not yet logged in.


Fatal Errors

At anypoint int the retrieval process you have to expect the following output:

  • 6xx INTERNAL SERVER ERROR
ERROR: {str errormessage}
OR
  • 6xx INTERNAL SERVER ERROR - {str errormessage}

Such errors should be reported back to Exp. (xx is a number between 00 and 99)

If you supply an unknown or unimplemented command you will get an error:

  • 598 UNKNOWN COMMAND
Retrieved from "https://wiki.anidb.net/index.php?title=UDP_API_Definition&oldid=2130"