UDP API DEV: Difference between revisions

From AniDB
Jump to navigation Jump to search
mNo edit summary
(Undo revision 16194 by (lol reading bold at the top of the page is hard, I'll submit a tracker item) Fansubcounter (talk))
 
(113 intermediate revisions by 21 users not shown)
Line 1: Line 1:
'''UDP DEF V0.03 DEV'''
{{TOCright}}
* chii powr / minichii
'''[[UDP_API_Definition|UDP API]] DEVELOPMENT'''
* feel free to edit / comment / whatever
* do use discussion
* [[User:Exp|Exp]] will approve / disapprove
* [[User:Epoximator|Epoximator]] will implement


== General ==
'''Please use the tracker for all feature requests and bug reports:''' http://tracker.anidb.net
* All commands except PING requires login, meaning session tag must be set (s=xxxxx).
* Possible return codes for all commands:
:* 600 INTERNAL SERVER ERROR
:* 601 ANIDB OUT OF SERVICE - TRY AGAIN LATER
:* 505 ILLEGAL INPUT OR ACCESS DENIED
:* 598 UNKNOWN COMMAND (sort of)
* Additional return codes for all commands that require login:
:* 501 LOGIN FIRST
:* 506 INVALID SESSION


* General return data:
This page is for laying out larger (or radical) additions / changes to the [[UDP API Definition|UDP API]] using the same format and style as the API documentation. This makes it easy to compare different solutions and to copy over changes to the documentation when finally implemented. Tracker items should suffice for minor requests and bug reports.
  {three digit return code} {str return string}\n
  {data field 0}|{data field 1}|...|{data field n}
* Exceptions are 200,201,271,272,504,? which returns additional data in the first line.
== Data ==
=== ANIME : new ===
'''Description:'''<br>
Retrieves most info available for a specific anime defined by either id or name.


'''Command String:'''<br>
Feel free to edit / comment / whatever.
by aid
* ANIME aid={int4 id}
by name
* ANIME aname={str anime name}
 
'''Possible Replies:'''
* XXX NO SUCH ANIME
* XXX ANIME
: {int4 aid}|{int4 eps}|{int4 ep count}|{int4 special cnt}|{int4 rating}|{int4 votes}|{int4 tmprating}|{int4 tmpvotes}|{int4 review rating average}|{int4 reviews}|{str year}|{str type}|{str romaji}|{str kanji}|{str english}|{str other}|{str short names}|{str synonyms}|{str genre list}
* XXX ANIME BEST MATCH
: ...
'''Info:'''
* Synonyms and short names are separated with '
* Genre names are separated with ',' and ordered by weight (desc).
* Categories?
* By title: returns the most likely anime, i.e. only one
* NOTE: The genre list is the first data to be truncated if needed. And then: synonym list, short name list. This applies for the FILE command too.


'''Examples:''' (html escaped code intended)
== Data Commands ==
  > ANIME aname=tmm&s=xxxxx
=== [[UDP_API_Definition#ANIME:_Retrieve_Anime_Data|ANIME]] ===
  < 230 ANIME
  161|52|50|0|715|57|777|35|816|1|2002-2003|TV|Tokyo Mew Mew|&#26481;&#20140;&#12511;&#12517;&#12454;&#12511;&#12517;&#12454;||||TMM'mew|Cat Girls
 
  > ANIME aname=&#12490;&#12523;&#12488;&s=xxxxx
  < 230 ANIME
  239|0|140|2|1000|10|855|3750|803|36|2002-2005|TV|Naruto|&#12490;&#12523;&#12488;||&#1504;&#1488;&#1512;&#1493;&#1496;&#1493;|NARUTO'&#1606;&#1575;&#1585;&#1608;&#1578;&#1608;|naruto tv'ntv|Action,Shounen,Past,...(cut)


'''Comments:'''<br>
* ANIME aid={int4 id}[&acode={int4}[&lang={str}]]
I'd like this command to cope with a couple of different encodings when passed a title, and be explicit over whether it finds only one result or does a best-match guess. --[[User:Rar|Rar]] 04:51, 30 Oct 2005 (CET)
* FILE fid={int4 id}[&fcode={int4}&acode={int4}[&lang={str}]]
: On second thought, is a wide search really wanted or is 'WHERE lower(name)=' good enough?. About encodings, do you care to elaborate? --[[User:Epoximator|Epoximator]] 18:24, 25 November 2005 (CET)


=== EPISODE : new ===
Bit 23 of <tt>acode</tt> (other name) will return language (official title) defined by <tt>lang</tt> where <tt>lang</tt> is a list of ISO 639-1 (?) codes (i.e. <tt>de,en</tt>) ordered by preference. The first existing language will be returned. Applies to 24 (short name list) and 25 (synonym list) also.
'''Description:'''<br>
Retrieves most info available for a specific episode defined by either id or anime name/id + epno.


'''Command String:'''<br>
An alternative would be to use the language options in the [[profile]].
by eid
* EPISODE eid={int4 eid}
by anime and episode number
* EPISODE aname={str anime name}&epno={int4 episode number}
* EPISODE aid={int4 anime id}&epno={int4 episode number}
'''Possible Replies:'''
* XXX NO SUCH EPISODE
* XXX EPISODE
: {int4 eid}|{int4 aid}|{int4 length}|{int4 rating}|{int4 votes}|{str epno}|{str eng}|{str romaji}|{str kanji}
'''Info:'''
* length is in minutes
* epno includes special character (only if special) and padding (only if normal)
'''Examples:''' (html escaped code intended)
  > EPISODE eid=1&s=xxxxx
  < 240 EPISODE
  1|1|24|400|4|01|Invasion|shinryaku|侵略
 
  > EPISODE aname=Seikai no Monshou&epno=2&s=xxxxx
  < 240 EPISODE
  2|1|24|750|2|02|Kin of the Stars|Hoshi-tachi no Kenzoku|星たちの眷族


=== GROUP : new ===
=== [[UDP_API_Definition#EPISODES:_Retrieve_Episodes_Data|EPISODES]] ===
'''Description:'''<br>
Feature request:
Retrieves most info available for a specific group defined by either id or name.


'''Command String:'''<br>
* EPISODES aid={int4 id}
by gid
return names of all episodes in anime
* GROUP gid={int4 gid}
by name/shortname
* GROUP gname={str group name}
'''Possible Replies:'''
* XXX NO SUCH GROUP
* XXX GROUP
: {int4 gid}|{int4 rating}|{int4 votes}|{int4 acount}|{int fcount}|{str name}|{str short}|{str irc}|{str url}
'''Examples:'''
  > GROUP gid=1&s=xxxxx
  < 250 GROUP
  1|621|28|29|222|Animehaven|AH|#animehaven@irc.enterthegame.com|http://www.theanimehaven.com
 
  > GROUP gname=a-l&s=xxxxx
  < 250 GROUP
  566|860|398|48|503|Anime-Legion|A-L|#anime-legion@irc.irchighway.net|http://www.anime-legion.net


=== FILE: extend ===
Possible Replies:
'''Description:'''<br>
* {str eng}|{str romaji}|{str kanji}|...
Retrieves most available info for a specific file defined by id, size + ed2k or anime + group + epno.


'''Command String:'''<br>
: such a command can't be added. The length limit for UDP packets would break it for most anime. And the question is whether a command like this is actually needed in the UDP API. [[User:Exp|Exp]] 07:40, 27 July 2007 (UTC)
by fid:
:: well, for now i'm stuck with using wget anime page and than parse it, which is bad, cos there's many unneeded data => more traffic, and changes in html layout breaks compatibility( [[User:iSage|iSage]] 15:26, 27 July 2007 (UTC)
* FILE fid={int4 id}[&fcode={int4}&acode={int4}]
::: that's really bad. Why do you need to do that? What are you actually trying to do? Usually you'd use a MyList export as base, import it and only use the UDP API to update the data. [[User:Exp|Exp]] 17:59, 27 July 2007 (UTC)
by size+ed2k hash:
:::: I need to get a list of English names of all or some episodes of some specified anime. and MyList is not a solution. well, if this is considered as rip off.. sorry then, I'll try to look for another solution  [[User:iSage|iSage]] 12:01, 28 July 2007 (UTC)
* FILE size={int4 size}&ed2k={str ed2khash}[&fcode={int4}&acode={int4}]
::: what for? We might be able to come up with a good solution for your issue, however, we'd need a full account (the big picture!) of what you want to do and why you want to do it. I.e. why do you need those episodes? What triggers the request? What kind of application are we talking about anyway? ... [[User:Exp|Exp]] 20:11, 28 July 2007 (UTC)
by anime, group and epno
:::: web-site with file archive. trigger is: add new anime (or edit)->assign AniDB id->cache episodes. hope you'll understand what i mean. [[User:iSage|iSage]] 06:58, 07 August 2007 (UTC)
* FILE aname={str anime name}&gname={str group name}&epno={int4 episode number}[&fcode={int4}&acode={int4}]
::: what's the URL of that page? [[User:Exp|Exp]] 08:28, 7 August 2007 (UTC)
* FILE aname={str anime name}&gid={int4 group id}&epno={int4 episode number}[&fcode={int4}&acode={int4}]
:::: http://aumi.ru --[[User:ISage|ISage]] 17:33, 9 August 2007 (UTC)
* FILE aid={int4 anime id}&gname={str group name}&epno={int4 episode number}[&fcode={int4}&acode={int4}]
* FILE aid={int4 anime id}&gid={int4 group id}&epno={int4 episode number}[&fcode={int4}&acode={int4}]


'''Possible Replies:'''
=== [[UDP_API_Definition#GROUP:_Retrieve_Group_Data|GROUP]] ===
* XXX FILE
{{missing}}
: {int4 fid}|{int4 aid}|{int4 eid}|{int4 gid}|{int4 lid}|{int4 state}|{int4 size}|{str ed2k}|{str default file name}
* XXX FILE
: {int4 fid}|...(data list)
* XXX NO SUCH FILE


'''Info:'''
=== [[UDP_API_Definition#FILE:_Retrieve_File_Data|FILE]] ===
* fcode and acode is integers where each bit corresponds to a data field related to the specified file (se below). The data list received is sorted in the same order as the tables (and fcode before acode). {f|a}code=-1 means retrieve all fields.
*FILE size={int8 size}&ed2k={str ed2khash}[&fcode={int4}&acode={int4}]


'''fcode:'''
Is the UDP API supporting other checksum formats? Would it be too much trouble to implement i.e. md5 checksums?
{| cellpadding="0" align="center"
: Needs database support, meaning exp has to grant it. --[[User:Epoximator|Epoximator]] 09:34, 9 July 2007 (UTC)
! width="40"|Bit
:: Uniqueness of all other hashes besides the ed2k hash is not enforced by AniDB. Meaning their use in a FILE lookup might cause some problems in case multiple files with that hash are in the db. [[User:Exp|Exp]] 09:41, 9 July 2007 (UTC)
! width="80"|Decimal
! width="140"|Data field
! width="50"|-
! width="40"|Bit
! width="80"|Decimal
! width="140"|Data field
|- style="background-color: #eee;"
|0 ||1 || not used || || 16 || 65536 || str dub language
|-
|1 ||2 || int4 aid || || 17 || 131072 || str sub language
|- style="background-color: #eee;"
|2 ||4 || int4 eid || || 18 || 262144 || str quality
|-
|3 ||8 || int4 gid ||  || 19 || 524288 || str source
|- style="background-color: #eee;"
|4 ||16 || int4 lid || || 20 || 1048576 || str audio codec
|-
|5 ||32 || not used || || 21 || 2097152 || int4 audio bitrate
|- style="background-color: #eee;"
|6 ||64 || not used || || 22 || 4194304 || str video codec
|-
|7 ||128 || not used || || 23 || 8388608 || int4 video bitrate
|- style="background-color: #eee;"
|8 ||256 || int2 status || || 24 || 16777216 || str video resolution
|-
|9 ||512 || int4 size || || 25 || 33554432 || str file type (extension)
|- style="background-color: #eee;"
|10 ||1024 || str ed2k || || 26 || 67108864 || int4 length in seconds
|-
|11 ||2048 || str md5 || || 27 || 134217728 || not used
|- style="background-color: #eee;"
|12 ||4096 || str sha1 || || 28 || 268435456 || not used
|-
|13 ||819 || str crc32 || || 29 || 536870912 || not used
|- style="background-color: #eee;"
|14 ||16384 || not used || || 30 || 1073741824 || not used
|-
|15 ||32768 || not used || || 31 || -2147483648 || not used
|}
'''acode:'''
{| cellpadding="0" align="center"
! width="40"|Bit
! width="80"|Decimal
! width="140"|Data field
! width="50"|-
! width="40"|Bit
! width="80"|Decimal
! width="140"|Data field
|- style="background-color: #eee;"
|0 ||1 || str group name || || 16 || 65536 || int4 anime total episodes
|-
|1 ||2 || str group short name || || 17 || 131072 || int4 last episode nr (highest, not special)
|- style="background-color: #eee;"
|2 ||4 || not used || || 18 || 262144 || str year
|-
|3 ||8 || not used ||  || 19 || 524288 || str type
|- style="background-color: #eee;"
|4 ||16 || not used || || 20 || 1048576 || str romaji name
|-
|5 ||32 || not used || || 21 || 2097152 || str kanji name
|- style="background-color: #eee;"
|6 ||64 || not used || || 22 || 4194304 || str english name
|-
|7 ||128 || not used || || 23 || 8388608 || str other name
|- style="background-color: #eee;"
|8 ||256 || str epno || || 24 || 16777216 || str short name list
|-
|9 ||512 || str ep name || || 25 || 33554432 || str synonym list
|- style="background-color: #eee;"
|10 ||1024 || str ep romaji name || || 26 || 67108864 || not genre list
|-
|11 ||2048 || str ep kanji name || || 27 || 134217728 || not used
|- style="background-color: #eee;"
|12 ||4096 || not used || || 28 || 268435456 || not used
|-
|13 ||819 || not used || || 29 || 536870912 || not used
|- style="background-color: #eee;"
|14 ||16384 || not used || || 30 || 1073741824 || not used
|-
|15 ||32768 || not used || || 31 || -2147483648 || not used
|}
'''Examples:''' (html escaped code intended)
<pre>
> FILE fid=15201&s=xxxxx
< 220 FILE
15201|74|445|41|1|242772540|a53c401ed95eaa502ba85acde773040c|Ai yori Aoshi - 1 - Relation - [Zhentarim DivX].ogm


> FILE fid=15201&fdata=-1&adata=-1&s=xxxxx
That's odd. How come AOM supports different hash-methods? (I know this may not be asked here) And shouldn't the md5sum be unique, too? I mean uniqueness IS the reason for using hash... Well I guess we can handle everything through ed2k though. [[User:Ainawing|Ainawing]] 15:49, 9 July 2007 (GMT+1)
< 220 FILE
: Well, the hashes are likely to be unique. However, this is not enforced by AniDB. Whether two distinct files really have the same hash or whether it is just an input error (i.e. wrong c&p) of the submitting user, the result is the same. You're trying to do a lookup for which you expect exactly one result or nothing at all and you end up with multiple results. And yes, most clients do support other hashing methods and also submit such data to AniDB. However, internally they are all based on ed2k hashes AFAIK. [[User:Exp|Exp]] 08:52, 10 July 2007 (UTC)
15201|74|445|41|0|1|242772540|a53c401ed95eaa502ba85acde773040c|a69c9ca88822338685f163db95da8231|842fc9060b4338757d0197a9f7bf0c79cf39a549|01ffc5f4|842fc9060b4338757d0197a9f7bf0c79cf39a549|dual (jap/eng)|english|very high|DVD|Ogg Vorbis|101|DivX5|1182|640x480|ogm|Zhentarim DivX|zx|24|24|2002|TV|Ai yori Aoshi|藍より青し|Bluer than Indigo|Azul||aya1'AYA'AiAo|
:: The real question is: Should we enforce uniqueness? It's not really a problem to implement and shouldn't lead to any problems. We have reports on uniqueness and there are ATM 2x2 files with equal md5, which obviously is wrong (if you look at them). Constraints would just mean that DerIdiot doesn't have to go around and fix such entries from time to time. On the other hand, supporting md5 lookup for clients is hardly important and it'll only have a negative impact on performance (although probably unnoticeable). --[[User:Epoximator|Epoximator]] 12:02, 10 July 2007 (UTC)
::: Well, enforcing uniqueness in the DB brings a slight performance overhead with it, but as file additions are a very seldom event, that wouldn't hurt us. We'd have a couple of extra indices on the file table, which would increase the storage requirements of the DB. Though even that wouldn't be all that much. Supporting MD5 hashes on the UDP API might simplify the writing of very simple UDP clients, as there are easily available MD5 libraries for every programming language out there. For ed2k/md4 libs you might have to search for a bit. However, I think I wouldn't go as far as to enforce uniqueness for all our hashes (i.e. sha1 and tth). But it might be seriously worth considering enforcing MD5 uniqueness, especially if we might some day drop ed2k hashes. MD5 hashes would offer a nice fallback in such a case. [[User:Exp|Exp]] 08:52, 11 July 2007 (UTC)
:::: Agreed. Even better, since hashes are only being used for identification and checksumming, and not authentication, there's no real need for uniqueness other than the 1 in several trillion chance of a collision. The odds get even better with SHA1 or Tiger or RIPEMD. ed2k is not exactly a standard hash, and MD4 is simply not present in some languages, unless you obtain an external library. (For example, I bundle a native Python MD4 with my client, since that library is rare outside of Linux distributions.) [[User:Billessig|Billessig]] 09:07, 26 July 2007 (UTC)
Well, how's it looking? There does not seem to be made any progress on this one. Has this been dropped or simply stalled? :-)  [[User:Ainawing|Ainawing]] 11:24, 19 November 2007 (CET)


> FILE aname=narutaru&gname=triad&amp;aone&epno=2&s=xxxxx
Feature Request:
< t001 220 FILE
* FILE aid={int4 anime id}&epno={int4 episode number}&size={int8 size}[&fcode={int4}&acode={int4}]
15459|782|8772|380|1|171298816|2c8a3b53d94d8579b9b81941c549e108|Narutaru - 02 - Catastrophe During the Daytime - [Triad & AonE].avi
Basically retrieve file information by anime, epno and size. In case the ed2k hash doesn't match and the file is thus apparently corrupt (and you're too lazy to figure that out yourself) --[[User:Tesiph|Tesiph]] 10:48, 17 December 2007 (CET)
</pre>
'''Comments:'''<br>
Pass a list of fields (limit e.g. 10 entries)
and return content. This makes it possible to request new fields in the database without
breaking API for older clients.
On updates, only the list of possible fields
need to be extendet. -- --[[User:Dinoex|Dinoex]]
: I think it's a good idea. Modified a bit. --[[User:Epoximator|Epoximator]] 13:38, 26 November 2005 (CET)


=== A4G : not planned ===
* Retrieve related-episode data with the FILE command, making use of bit 5 (or some other unused bit) of the fcode.  Proposed return is a list of episode IDs separated by commas.  File 275580 is a fair example of why this is useful; right now there's no way to show that this is a dual-episode file via the UDP API. --[[User:Radicand|Radicand]] 14:32, 27 March 2008 (UTC)
=== G4A : not planned ===
=== STITLE : not planned ===


== Mylist General ==
== MyList Commands ==
=== MYLISTADD: Add file to MyList ===
Feature request: It'd be useful if the reply of this command would contain the lid of the added file.


=== MYLIST : extend ===
'''Command String:'''<br>
by anime + group + epno
* MYLIST aname={str anime name}&gname={str group name}&epno={int epno}
'''Possible Replies:'''<br>
when multiple files found
* XXX MULTIPLE FILES FOUND
: {str eps with state unknown}|{str eps with state on hhd}|{str eps with state on cd}|{str eps with state deleted}|{str watched eps}|{str group 1 short name}|{str eps for group 1}|...|{str group N short name}|{str eps for group N}
'''Info:'''
* eps is a list of episodes, e.g. "1-12,14-16,T1"
'''Example:'''
  > MYLIST aname=gits sac&s=xxxxx
  <322 MULTIPLE FILES FOUND
  |1-26,S2-S27|1-26|||V-A|S2-S27|LMF|20-26|KAA|1-12,21-23|anime_fin|1-26|AonE|1-19|Anime-MX|1-3,17-20
=== MYLISTSTATS : new ===
'''Description:'''<br>
Retrieves the most relevant figures related to your user account.
'''Command String:'''
* MYLISTSTATS
'''Possible Replies:'''
'''Possible Replies:'''
* XXX MYLIST STATS
* 210 MYLIST ENTRY ADDED
{animes}|{eps}|{files}|{size of files}|{added animes}|{added eps}|{added files}|{added groups}|{leech %}|{lame %}|{viewed % of db}|{mylist % of db}|{viewed % of mylist}|{number of viewed eps}|{votes}|{reviews}
: {int4 number of entries added}[|{int4 lid}]


'''Info:'''
''Replies with 3xx and 4xx codes skipped.''
* all fields are int


=== MYLISTLATEST : not planned ===
'''Notes:'''
* The lid of the added file is written in the reply only in case just one file was added. ''Actually, if many files were added, there lid's would by useless to client because it wouldn't know the correspondence between lid's and files.''


=== VOTE : new ===
== Notify Commands ==
'''Description:'''<br>
{{missing}}
Votes for a specifed anime, episode or group defined by id or name (epno).


'''Command String:'''<br>
== Misc Commands ==
by id
=== ENCRYPT : Encrypt Command ===
* VOTE type={int2 type}&id={int4 id}[&value={int4 vote value}&epno={int4 episode number}]
This is a major modification of the UDP API encryption scheme which will break compatibility with
by name
older clients. However, the current system has some weaknesses and only few clients support encryption so far.
* VOTE type={int2 type}&name={string name}[&value={int4 vote value}&epno={int4 episode number}]
So this seems acceptable.


'''Possible Replies:'''
* AES 128bit, '''CBC mode''', PKCS5 padding
* XXX VOTED
** 128bit MD5 sum (raw, don't convert to hex) of API encryption key (API pass) is used as 128bit key.
: {str aname/ename/gname}
** 128bit MD5 sum (raw, don't convert to hex) of {str salt} in the reply of the ENCRYPT command is used as 128bit IV.
* XXX VOTE FOUND
: {str aname/ename/gname}|{vote value}
* XXX VOTE UPDATED
: {str aname/ename/gname}|{old vote value}
* XXX VOTE REVOKED
: {str aname/ename/gname}|{revoked vote value}
* XXX NO SUCH VOTE
* XXX INVALID VOTE TYPE
* XXX INVALID VOTE VALUE
* XXX PERMVOTE NOT ALLOWED
: {str aname}
* XXX ALREADY PERMVOTED
: {str aname/ename/gname}


'''Info:'''
* a specific binary preamble is added to all encrypted packets (sent by the client or by the server).
* type: 1=anime, 2=anime tmpvote, 3=group
** i.e. all encrypted packets start with the three bytes \0\1\0 (hex: 00 01 00).
* for episode voting add epno on type=1
* value: negative number means revoke, 0 means retrieve (default), 100-1000 are valid vote values, rest is illegal
* votes will be updated automatically (no questions asked)
* tmpvoting when there exist a perm vote is not possible


=== RANDOM : new ===
* if encryption is enabled, check every incoming packet for preamble.
'''Command String:'''
** if preamble is present: decrypt and process packet. Reply is also encrypted.
* RANDOMANIME type={int4 type}
*** do not automatically disable encryption if the server fails to decrypt a packet, instead discard the packet. Send an unencrypted error reply "XXX DECRYPTION FAILED - PACKET IGNORED".
** otherwise:
*** if it is an ENCRYPT packet, disable encryption, discard the old key and iv and then process the ENCRYPT packet (thus re-enabling encryption with a new key and iv)
*** for any other type of unencrypted packet, discard the packet. Send an unencrypted error reply "XXX ENCRYPTION ENABLED - UNENCRYPTED PACKET IGNORED"


'''Possible Replies:'''
* ENCRYPT command:
* XXX ANIME ... (see ANIME)
** add type=0 to disable encryption (making it possible to disable encryption via "ENCRYPT type=0" once it was enabled) (such a command could be sent either encrypted or unencrypted).
** type=0 does an implicit logout (prevents 3rd parties from disabling server encryption, i.e. to sniff plain text notification data).
** type!=0 does an implicit logout if (and only if!) the user parameter differs from the currently logged in user for that connection.


'''Info:'''
any more ideas?
* type: 0=from db, 1=watched, 2=unwatched, 3=all mylist


=== CREQ : not planned ===
General goals of this change are:
* use better supported and more secure CBC mode
* make debugging of encryption problems easier for client developers (plain text error replies, allow disabling of encryption, resending ENCRYPT, preamble)
* make it always clear whether a packet is encrypted or plain text (preamble)
* allow clients to recover from encryption failures without having to wait for the connection to timeout (resending ENCRYPT)
* prevent the server from unintentionally sending unencrypted data to a client, i.e. notifications. (no more dropping of encryption on decryption failure)


== Mylist Edit ==
=== USERLIST : Retrieve MyList Data of a specified user [{{colour|red|rejected}}] ===
Feature Request: The purpose is for a client I want to make, which will essentially be a more user friendly method of viewing and manipulating the view of a user's list. Sort by various categories and so on. I would like to be able to view and browse a specified user's list just as I can with the web interface


=== MYLISTADD : extend ===
Probably best implemented to behave just like the MyList commands.
'''Command String:'''<br>
by anime + group + epno
* MYLISTADD aname={str anime name}[&gname={str group name}&epno={int4 episode number}][...]
* MYLISTADD aname={str anime name}[&gid={int4 group id}&epno={int4 episode number}][...]
* MYLISTADD aid={int4 anime id}[&gname={str group name}&epno={int4 episode number}][...]
* MYLISTADD aid={int4 anime id}[&gid={int4 group id}&epno={int4 episode number}][&state={int2 state}&viewed={boolean viewed}&source={str source}&storage={str storage}&other={str other}][&edit=1]
'''Possible Replies:'''<br>
when edit=1
* XXX MYLIST ENTRY EDITED
: {int4 number of entries edited}
when edit=0
* XXX MYLIST ENTRY ADDED
: {int4 number of entries added}
* XXX MULTIPLE FILES FOUND
: {int4 fid 1}|{int4 group name 1}
: ...
: {int4 fid n}|{int4 group name n}
'''Info:'''
* epno=0 means all eps (default), negative numbers means upto. (-12 -> upto 12)
* group is optional only when edit=1, meaning you can't add every file for an anime


=== MYLISTDEL : extend ===
'''Command String:'''<br>
'''Command String:'''<br>
by lid: (userlist user={str name}&id)
* USERLIST user={str name}&lid={int4 lid}
by fid:
* USERLIST user={str name}&fid={int4 fid}
by size+ed2k hash:
* USERLIST user={str name}&size={int4 size}&ed2k={str ed2khash}
by anime + group + epno
by anime + group + epno
* MYLISTDEL aname={str anime name}[&gname={str group name}&epno={int4 episode number}]
* USERLIST user={str name}&aname={str anime name}&gname={str group name}&epno={int4 episode number}
* MYLISTDEL aname={str anime name}[&gid={int4 group id}&epno={int4 episode number}]
* USERLIST user={str name}&aname={str anime name}&gid={int4 group id}&epno={int4 episode number}
* MYLISTDEL aid={int4 anime id}[&gname={str group name}&epno={int4 episode number}]
* USERLIST user={str name}&aid={int4 anime id}&gname={str group name}&epno={int4 episode number}
* MYLISTDEL aid={int4 anime id}[&gid={int4 group id}&epno={int4 episode number}]
* USERLIST user={str name}&aid={int4 anime id}&gid={int4 group id}&epno={int4 episode number}
'''Possible Replies:'''<br>
* XXX ENTRY DELETED
: {int4 number of entries}
'''Info:'''
* epno=0 means all eps (default), negative numbers means upto. (-12 -> upto 12)
* group is optional
* command will delete all enties found
 
== Misc ==
 
=== STATS : new ===
'''Command String:'''
* STATS
 
'''Possible Replies:'''
* XXX STATS
: {int4 animes)|{int4 eps}|{int4 files}|{int4 groups}|{int4 users}|{int4 creqs}
 
=== UPTIME : new ===
'''Command String:'''
* UPTIME
 
'''Possible Replies:'''
* XXX UPTIME
: {int4 udpserver uptime in sec}
 
=== TOP : new ===
'''Command String:'''
* TOP


'''Possible Replies:'''
'''Possible Replies:'''
* XXX TOP
* 221 USERLIST
: {str longest mylist}|{int count}
: {int4 lid}|{int4 fid}|{int4 eid}|{int4 aid}|{int4 gid}|{int4 date}|{int2 state}|{int4 viewdate}|{str storage}|{str source}|{str other}
: {str largest mylist}|{int count}
* 322 MULTIPLE FILES FOUND
: {str most lame files}|{int count}
: {str anime title}|{int episodes}|{str eps with state unknown}|{str eps with state on hhd}|{str eps with state on cd}|{str eps with state deleted}|{str watched eps}|{str group 1 short name}|{str eps for group 1}|...|{str group N short name}|{str eps for group N}
: {str most indep. user}|{int count}
* 321 NO SUCH ENTRY
: {str biggest leecher}|{int count}
: {str most anime added}|{int count}
: {str most eps added}|{int count}
: {str most files added}|{int count}
: {str most groups added}|{int count}
: {str most votes}|{int count}
: {str most reviews}|{int count}


'''Info:'''
'''Info:'''
* 'Hide myself in IRC stats' applies for this too.
* The state field provides information about the location and sharing state of a file in MyList.
* If files are added after hashing, a client should specify the state as 1 (on hdd) (if the user doesn't explicitly select something else).
* eps is a list of episodes, e.g. "1-12,14-16,T1"


=== LATEST : not planned ===
'''States:'''
  0 - unknown - state is unknown or the user doesn't want to provide this information
  1 - on hdd - the file is stored on HDD (but is not shared)
  2 - on cd - the file is stored on CD
  3 - deleted - the file has been deleted or is not available for other reasons (i.e. re-encoded)


== RETURN CODES ==
'''Example:'''
:* C : current, do not change
  > USERLIST user=baka&aname=gits sac&s=xxxxx
:* N : new
  < 322 MULTIPLE FILES FOUND
  Koukaku Kidoutai STAND ALONE COMPLEX|26||1-26|1-26,S2-S27|||V-A|S2-S27|LMF|20-26|KAA|1-26|AonE|1-19|Anime-MX|1-3,9-20


<pre>
'''Comments:'''<br>
/*
It would probably be ok to add this, but note that it would be a hassle to fetch a complete MyList. I mean, you can already do this with your own list, but it takes a lot of time because of the datagram rate limit. (It would take at least 8h to fetch my own list). And that is usage we don't really want @ UDP API. And a command that returns all the data at once will, of course, not be allowed. So, in other words, if you just want to check if a friend has a specific file, or what files per anime (multiple files found), then OK, but ''view and browse a specified user's list just as I can with the web interface'' is not likely to happen/be allowed.--[[User:Epoximator|Epoximator]] 11:29, 4 June 2006 (CEST)
* POSITVE 2XX
:What I'm really interested in is maintaining local dbs of 3/4 user lists and having a client automatically update them every 2/3 weeks. We use this to coordinate information within our country regarding who has what =p. But what we're interested in is a way to be able to selectively retrieve the entire list by some index. Of course if its slow, then it makes sense to maintain it locally and just look for changes. The other thing I really want is to somehow retrieve with MyList file data (or user list file data), the date at which something was added to the list. (I understand this information exists). The idea here is we can locally query these lists for files added within the last 2/3 months. --[[User:path|path]] 11:53, 25 June 2006 (IST)
*/
::That's really not wanted usage for this API. Have you thought about the possibility to use MyList export? You know you can make you own export template and get all the info you want at once? A command that query all changes (lids) within the last 2/3 months (or even weeks) is not going to be added. --[[User:Epoximator|Epoximator]] 12:36, 25 June 2006 (UTC)
LOGIN_ACCEPTED =200, //C
LOGIN_ACCEPTED_NEW_VER =201, //C
LOGGED_OUT =203, //C
STATS =206, //N
TOP =207, //N
UPTIME =208, //N
 
MYLIST_ENTRY_ADDED =210, //C
ENTRY_DELETED =211, //C
 
FILE =220, //C
MYLIST =221, //C
MYLIST_STATS =222, //N
 
ANIME =230, //N
ANIME_BEST_MATCH =231, //N
RANDOMANIME =232, //N
 
EPISODE =240, //N
GROUP =250, //N
 
VOTED =260, //N
VOTE_FOUND =261, //N
VOTE_UPDATED =262, //N
VOTE_REVOKED =263, //N
 
NOTIFICATION_ENABLED =270, //C
PUSHACK_CONFIRMED =280, //C
NOTIFYACK_SUCCESSFUL_M =281, //C
NOTIFYACK_SUCCESSFUL_N =282, //C
NOTIFICATION =290, //C
NOTIFYLIST =291, //C
NOTIFYGET_MESSAGE =292, //C
NOTIFYGET_NOTIFY =293, //C
SENDMSG_SUCCESSFUL =294, //C
 
/*
* AFFIRMATIVE/NEGATIVE 3XX
*/
PONG =300, //C
FILE_ALREADY_IN_MYLIST =310, //C
MYLIST_ENTRY_EDITED =311, //C
NO_SUCH_FILE =320, //C
NO_SUCH_ENTRY =321, //C
MULTIPLE_FILES_FOUND =322, //N
 
NO_SUCH_ANIME =330, //N
NO_SUCH_EPISODE =340, //N
NO_SUCH_GROUP =350, //N
 
NO_SUCH_VOTE =360, //N
INVALID_VOTE_TYPE =361, //N
INVALID_VOTE_VALUE =362, //N
PERMVOTE_NOT_ALLOWED =363, //N
ALREADY_PERMVOTED =364, //N
 
NOTIFICATION_DISABLED =370, //C
NO_SUCH_PACKET_PENDING =380, //C
NO_SUCH_ENTRY_M =381, //C
NO_SUCH_ENTRY_N =382, //C
 
NO_SUCH_MESSAGE =392, //C
NO_SUCH_NOTIFY =393, //C
NO_SUCH_USER =394, //C
 
NO_SUCH_DATA_ENTRY =396, //N
 
/*
* NEGATIVE 4XX
*/
 
NOT_LOGGED_IN =403, //C
NO_SUCH_MYLIST_FILE =410, //C
NO_SUCH_MYLIST_ENTRY =411, //C
 
 
/*
* CLIENT SIDE FAILURE 5XX
*/


LOGIN_FAILED =500, //C
== Other ==
LOGIN_FIRST =501, //C
=== Web Service API ===
ACCESS_DENIED =502, //C
I think most of the UDP API functionality could be exposed as a web service. Not only it would  make client programming simpler but could also solve some NAT/Proxy/Firewall UDP problems. This web service would only provide query capabilities, since callbacks (required for notifications) in web services still pose some problems. --[[User:Jassuncao|Jassuncao]] 08:43, 20 September 2006 (UTC)
CLIENT_VERSION_OUTDATED =503, //C
CLIENT_BANNED =504, //C
ILLEGAL_INPUT_OR_ACCESS_DENIED =505, //C
INVALID_SESSION =506, //C
BANNED_FOR_FLOODING =507, //C
UNKNOWN_COMMAND =598, //C


/*
:Web services are for big idiot corps with rooms of servers and no competent programmers. Spend TCP->HTTP->SOAP overhead to save on personnel. --[[User:Rar|Rar]] 21:49, 23 September 2006 (UTC)
* SERVER SIDE FAILURE 6XX
*/


INTERNAL_SERVER_ERROR =600, //C
::Since this is not the right place to start a flame war I will not comment.
ANIDB_OUT_OF_SERVICE =601, //C
API_VIOLATION =666, //C
</pre>


=== HTTP/XML API===
A complementary [[HTTP API Definition|HTTP/XML API]] has been suggested (and approved). See http://forum.anidb.net/viewtopic.php?f=11&t=6871


[[Category:Developement]]
[[Category:Development]]
[[Category:UDP]]
[[Category:API]]

Latest revision as of 06:07, 18 December 2011

UDP API DEVELOPMENT

Please use the tracker for all feature requests and bug reports: http://tracker.anidb.net

This page is for laying out larger (or radical) additions / changes to the UDP API using the same format and style as the API documentation. This makes it easy to compare different solutions and to copy over changes to the documentation when finally implemented. Tracker items should suffice for minor requests and bug reports.

Feel free to edit / comment / whatever.

Data Commands

ANIME

  • ANIME aid={int4 id}[&acode={int4}[&lang={str}]]
  • FILE fid={int4 id}[&fcode={int4}&acode={int4}[&lang={str}]]

Bit 23 of acode (other name) will return language (official title) defined by lang where lang is a list of ISO 639-1 (?) codes (i.e. de,en) ordered by preference. The first existing language will be returned. Applies to 24 (short name list) and 25 (synonym list) also.

An alternative would be to use the language options in the profile.

EPISODES

Feature request:

  • EPISODES aid={int4 id}

return names of all episodes in anime

Possible Replies:

  • {str eng}|{str romaji}|{str kanji}|...
such a command can't be added. The length limit for UDP packets would break it for most anime. And the question is whether a command like this is actually needed in the UDP API. Exp 07:40, 27 July 2007 (UTC)
well, for now i'm stuck with using wget anime page and than parse it, which is bad, cos there's many unneeded data => more traffic, and changes in html layout breaks compatibility( iSage 15:26, 27 July 2007 (UTC)
that's really bad. Why do you need to do that? What are you actually trying to do? Usually you'd use a MyList export as base, import it and only use the UDP API to update the data. Exp 17:59, 27 July 2007 (UTC)
I need to get a list of English names of all or some episodes of some specified anime. and MyList is not a solution. well, if this is considered as rip off.. sorry then, I'll try to look for another solution iSage 12:01, 28 July 2007 (UTC)
what for? We might be able to come up with a good solution for your issue, however, we'd need a full account (the big picture!) of what you want to do and why you want to do it. I.e. why do you need those episodes? What triggers the request? What kind of application are we talking about anyway? ... Exp 20:11, 28 July 2007 (UTC)
web-site with file archive. trigger is: add new anime (or edit)->assign AniDB id->cache episodes. hope you'll understand what i mean. iSage 06:58, 07 August 2007 (UTC)
what's the URL of that page? Exp 08:28, 7 August 2007 (UTC)
http://aumi.ru --ISage 17:33, 9 August 2007 (UTC)

GROUP

The description is missing or severely incomplete.
If you can, please help by explaining it.

FILE

  • FILE size={int8 size}&ed2k={str ed2khash}[&fcode={int4}&acode={int4}]

Is the UDP API supporting other checksum formats? Would it be too much trouble to implement i.e. md5 checksums?

Needs database support, meaning exp has to grant it. --Epoximator 09:34, 9 July 2007 (UTC)
Uniqueness of all other hashes besides the ed2k hash is not enforced by AniDB. Meaning their use in a FILE lookup might cause some problems in case multiple files with that hash are in the db. Exp 09:41, 9 July 2007 (UTC)

That's odd. How come AOM supports different hash-methods? (I know this may not be asked here) And shouldn't the md5sum be unique, too? I mean uniqueness IS the reason for using hash... Well I guess we can handle everything through ed2k though. Ainawing 15:49, 9 July 2007 (GMT+1)

Well, the hashes are likely to be unique. However, this is not enforced by AniDB. Whether two distinct files really have the same hash or whether it is just an input error (i.e. wrong c&p) of the submitting user, the result is the same. You're trying to do a lookup for which you expect exactly one result or nothing at all and you end up with multiple results. And yes, most clients do support other hashing methods and also submit such data to AniDB. However, internally they are all based on ed2k hashes AFAIK. Exp 08:52, 10 July 2007 (UTC)
The real question is: Should we enforce uniqueness? It's not really a problem to implement and shouldn't lead to any problems. We have reports on uniqueness and there are ATM 2x2 files with equal md5, which obviously is wrong (if you look at them). Constraints would just mean that DerIdiot doesn't have to go around and fix such entries from time to time. On the other hand, supporting md5 lookup for clients is hardly important and it'll only have a negative impact on performance (although probably unnoticeable). --Epoximator 12:02, 10 July 2007 (UTC)
Well, enforcing uniqueness in the DB brings a slight performance overhead with it, but as file additions are a very seldom event, that wouldn't hurt us. We'd have a couple of extra indices on the file table, which would increase the storage requirements of the DB. Though even that wouldn't be all that much. Supporting MD5 hashes on the UDP API might simplify the writing of very simple UDP clients, as there are easily available MD5 libraries for every programming language out there. For ed2k/md4 libs you might have to search for a bit. However, I think I wouldn't go as far as to enforce uniqueness for all our hashes (i.e. sha1 and tth). But it might be seriously worth considering enforcing MD5 uniqueness, especially if we might some day drop ed2k hashes. MD5 hashes would offer a nice fallback in such a case. Exp 08:52, 11 July 2007 (UTC)
Agreed. Even better, since hashes are only being used for identification and checksumming, and not authentication, there's no real need for uniqueness other than the 1 in several trillion chance of a collision. The odds get even better with SHA1 or Tiger or RIPEMD. ed2k is not exactly a standard hash, and MD4 is simply not present in some languages, unless you obtain an external library. (For example, I bundle a native Python MD4 with my client, since that library is rare outside of Linux distributions.) Billessig 09:07, 26 July 2007 (UTC)

Well, how's it looking? There does not seem to be made any progress on this one. Has this been dropped or simply stalled? :-) Ainawing 11:24, 19 November 2007 (CET)

Feature Request:

  • FILE aid={int4 anime id}&epno={int4 episode number}&size={int8 size}[&fcode={int4}&acode={int4}]

Basically retrieve file information by anime, epno and size. In case the ed2k hash doesn't match and the file is thus apparently corrupt (and you're too lazy to figure that out yourself) --Tesiph 10:48, 17 December 2007 (CET)

  • Retrieve related-episode data with the FILE command, making use of bit 5 (or some other unused bit) of the fcode. Proposed return is a list of episode IDs separated by commas. File 275580 is a fair example of why this is useful; right now there's no way to show that this is a dual-episode file via the UDP API. --Radicand 14:32, 27 March 2008 (UTC)

MyList Commands

MYLISTADD: Add file to MyList

Feature request: It'd be useful if the reply of this command would contain the lid of the added file.

Possible Replies:

  • 210 MYLIST ENTRY ADDED
{int4 number of entries added}[|{int4 lid}]

Replies with 3xx and 4xx codes skipped.

Notes:

  • The lid of the added file is written in the reply only in case just one file was added. Actually, if many files were added, there lid's would by useless to client because it wouldn't know the correspondence between lid's and files.

Notify Commands

The description is missing or severely incomplete.
If you can, please help by explaining it.

Misc Commands

ENCRYPT : Encrypt Command

This is a major modification of the UDP API encryption scheme which will break compatibility with older clients. However, the current system has some weaknesses and only few clients support encryption so far. So this seems acceptable.

  • AES 128bit, CBC mode, PKCS5 padding
    • 128bit MD5 sum (raw, don't convert to hex) of API encryption key (API pass) is used as 128bit key.
    • 128bit MD5 sum (raw, don't convert to hex) of {str salt} in the reply of the ENCRYPT command is used as 128bit IV.
  • a specific binary preamble is added to all encrypted packets (sent by the client or by the server).
    • i.e. all encrypted packets start with the three bytes \0\1\0 (hex: 00 01 00).
  • if encryption is enabled, check every incoming packet for preamble.
    • if preamble is present: decrypt and process packet. Reply is also encrypted.
      • do not automatically disable encryption if the server fails to decrypt a packet, instead discard the packet. Send an unencrypted error reply "XXX DECRYPTION FAILED - PACKET IGNORED".
    • otherwise:
      • if it is an ENCRYPT packet, disable encryption, discard the old key and iv and then process the ENCRYPT packet (thus re-enabling encryption with a new key and iv)
      • for any other type of unencrypted packet, discard the packet. Send an unencrypted error reply "XXX ENCRYPTION ENABLED - UNENCRYPTED PACKET IGNORED"
  • ENCRYPT command:
    • add type=0 to disable encryption (making it possible to disable encryption via "ENCRYPT type=0" once it was enabled) (such a command could be sent either encrypted or unencrypted).
    • type=0 does an implicit logout (prevents 3rd parties from disabling server encryption, i.e. to sniff plain text notification data).
    • type!=0 does an implicit logout if (and only if!) the user parameter differs from the currently logged in user for that connection.

any more ideas?

General goals of this change are:

  • use better supported and more secure CBC mode
  • make debugging of encryption problems easier for client developers (plain text error replies, allow disabling of encryption, resending ENCRYPT, preamble)
  • make it always clear whether a packet is encrypted or plain text (preamble)
  • allow clients to recover from encryption failures without having to wait for the connection to timeout (resending ENCRYPT)
  • prevent the server from unintentionally sending unencrypted data to a client, i.e. notifications. (no more dropping of encryption on decryption failure)

USERLIST : Retrieve MyList Data of a specified user [rejected]

Feature Request: The purpose is for a client I want to make, which will essentially be a more user friendly method of viewing and manipulating the view of a user's list. Sort by various categories and so on. I would like to be able to view and browse a specified user's list just as I can with the web interface

Probably best implemented to behave just like the MyList commands.

Command String:
by lid: (userlist user={str name}&id)

  • USERLIST user={str name}&lid={int4 lid}

by fid:

  • USERLIST user={str name}&fid={int4 fid}

by size+ed2k hash:

  • USERLIST user={str name}&size={int4 size}&ed2k={str ed2khash}

by anime + group + epno

  • USERLIST user={str name}&aname={str anime name}&gname={str group name}&epno={int4 episode number}
  • USERLIST user={str name}&aname={str anime name}&gid={int4 group id}&epno={int4 episode number}
  • USERLIST user={str name}&aid={int4 anime id}&gname={str group name}&epno={int4 episode number}
  • USERLIST user={str name}&aid={int4 anime id}&gid={int4 group id}&epno={int4 episode number}

Possible Replies:

  • 221 USERLIST
{int4 lid}|{int4 fid}|{int4 eid}|{int4 aid}|{int4 gid}|{int4 date}|{int2 state}|{int4 viewdate}|{str storage}|{str source}|{str other}
  • 322 MULTIPLE FILES FOUND
{str anime title}|{int episodes}|{str eps with state unknown}|{str eps with state on hhd}|{str eps with state on cd}|{str eps with state deleted}|{str watched eps}|{str group 1 short name}|{str eps for group 1}|...|{str group N short name}|{str eps for group N}
  • 321 NO SUCH ENTRY

Info:

  • The state field provides information about the location and sharing state of a file in MyList.
  • If files are added after hashing, a client should specify the state as 1 (on hdd) (if the user doesn't explicitly select something else).
  • eps is a list of episodes, e.g. "1-12,14-16,T1"

States:

 0 - unknown - state is unknown or the user doesn't want to provide this information
 1 - on hdd - the file is stored on HDD (but is not shared)
 2 - on cd - the file is stored on CD
 3 - deleted - the file has been deleted or is not available for other reasons (i.e. re-encoded)

Example:

 > USERLIST user=baka&aname=gits sac&s=xxxxx
 < 322 MULTIPLE FILES FOUND
 Koukaku Kidoutai STAND ALONE COMPLEX|26||1-26|1-26,S2-S27|||V-A|S2-S27|LMF|20-26|KAA|1-26|AonE|1-19|Anime-MX|1-3,9-20

Comments:
It would probably be ok to add this, but note that it would be a hassle to fetch a complete MyList. I mean, you can already do this with your own list, but it takes a lot of time because of the datagram rate limit. (It would take at least 8h to fetch my own list). And that is usage we don't really want @ UDP API. And a command that returns all the data at once will, of course, not be allowed. So, in other words, if you just want to check if a friend has a specific file, or what files per anime (multiple files found), then OK, but view and browse a specified user's list just as I can with the web interface is not likely to happen/be allowed.--Epoximator 11:29, 4 June 2006 (CEST)

What I'm really interested in is maintaining local dbs of 3/4 user lists and having a client automatically update them every 2/3 weeks. We use this to coordinate information within our country regarding who has what =p. But what we're interested in is a way to be able to selectively retrieve the entire list by some index. Of course if its slow, then it makes sense to maintain it locally and just look for changes. The other thing I really want is to somehow retrieve with MyList file data (or user list file data), the date at which something was added to the list. (I understand this information exists). The idea here is we can locally query these lists for files added within the last 2/3 months. --path 11:53, 25 June 2006 (IST)
That's really not wanted usage for this API. Have you thought about the possibility to use MyList export? You know you can make you own export template and get all the info you want at once? A command that query all changes (lids) within the last 2/3 months (or even weeks) is not going to be added. --Epoximator 12:36, 25 June 2006 (UTC)

Other

Web Service API

I think most of the UDP API functionality could be exposed as a web service. Not only it would make client programming simpler but could also solve some NAT/Proxy/Firewall UDP problems. This web service would only provide query capabilities, since callbacks (required for notifications) in web services still pose some problems. --Jassuncao 08:43, 20 September 2006 (UTC)

Web services are for big idiot corps with rooms of servers and no competent programmers. Spend TCP->HTTP->SOAP overhead to save on personnel. --Rar 21:49, 23 September 2006 (UTC)
Since this is not the right place to start a flame war I will not comment.

HTTP/XML API

A complementary HTTP/XML API has been suggested (and approved). See http://forum.anidb.net/viewtopic.php?f=11&t=6871