UDP API DEV: Difference between revisions

m
no edit summary
mNo edit summary
mNo edit summary
Line 16: Line 16:
* FILE fid={int4 id}[&fcode={int4}&acode={int4}[&lang={str}]]
* FILE fid={int4 id}[&fcode={int4}&acode={int4}[&lang={str}]]


Bit 23 of <tt>acode</tt> (other name) will return language (official title) defined by <tt>lang</tt> where <tt>lang</tt> is a list of ISO 639-1 (?) codes (ie. <tt>de,en</tt>) ordered by preference. The first existing language will be returned. Applies to 24 (short name list) and 25 (synonym list) also.
Bit 23 of <tt>acode</tt> (other name) will return language (official title) defined by <tt>lang</tt> where <tt>lang</tt> is a list of ISO 639-1 (?) codes (i.e. <tt>de,en</tt>) ordered by preference. The first existing language will be returned. Applies to 24 (short name list) and 25 (synonym list) also.


An alternative would be to use the language options in the [[profile]].
An alternative would be to use the language options in the [[profile]].
Line 30: Line 30:


: such a command can't be added. The length limit for UDP packets would break it for most anime. And the question is whether a command like this is actually needed in the UDP API. [[User:Exp|Exp]] 07:40, 27 July 2007 (UTC)
: such a command can't be added. The length limit for UDP packets would break it for most anime. And the question is whether a command like this is actually needed in the UDP API. [[User:Exp|Exp]] 07:40, 27 July 2007 (UTC)
:: well, for now i'm stuck with using wget anime page and than parse it, which is bad, cos theres many unneeded data => more traffic, and changes in html layout breaks compatibility( [[User:iSage|iSage]] 15:26, 27 July 2007 (UTC)
:: well, for now i'm stuck with using wget anime page and than parse it, which is bad, cos there's many unneeded data => more traffic, and changes in html layout breaks compatibility( [[User:iSage|iSage]] 15:26, 27 July 2007 (UTC)
::: that's really bad. Why do you need to do that? What are you actually trying to do? Usually you'd use a mylist export as base, import it and only use the UDP API to update the data. [[User:Exp|Exp]] 17:59, 27 July 2007 (UTC)
::: that's really bad. Why do you need to do that? What are you actually trying to do? Usually you'd use a MyList export as base, import it and only use the UDP API to update the data. [[User:Exp|Exp]] 17:59, 27 July 2007 (UTC)
:::: i'm need to get list of english names of all or some episodes of some specified anime. and mylist is not a solution. well, if this is considered as ripoff.. sorry then, i'll try to look for another solution  [[User:iSage|iSage]] 12:01, 28 July 2007 (UTC)
:::: I need to get a list of English names of all or some episodes of some specified anime. and MyList is not a solution. well, if this is considered as rip off.. sorry then, I'll try to look for another solution  [[User:iSage|iSage]] 12:01, 28 July 2007 (UTC)
::: what for? We might be able to come up with a good solution for your issue, however, we'd need a full account (the big picture!) of what you want to do and why you want to do it. I.e. why do you need those episodes? What triggers the request? What kind of application are we talking about anyway? ... [[User:Exp|Exp]] 20:11, 28 July 2007 (UTC)
::: what for? We might be able to come up with a good solution for your issue, however, we'd need a full account (the big picture!) of what you want to do and why you want to do it. I.e. why do you need those episodes? What triggers the request? What kind of application are we talking about anyway? ... [[User:Exp|Exp]] 20:11, 28 July 2007 (UTC)
:::: web-site with file archive. triger is: add new anime (or edit)->assign AniDB id->cache episodes. hope you'll understand what i mean. [[User:iSage|iSage]] 06:58, 07 August 2007 (UTC)
:::: web-site with file archive. trigger is: add new anime (or edit)->assign AniDB id->cache episodes. hope you'll understand what i mean. [[User:iSage|iSage]] 06:58, 07 August 2007 (UTC)
::: what's the url of that page? [[User:Exp|Exp]] 08:28, 7 August 2007 (UTC)
::: what's the URL of that page? [[User:Exp|Exp]] 08:28, 7 August 2007 (UTC)
:::: http://aumi.ru --[[User:ISage|ISage]] 17:33, 9 August 2007 (UTC)
:::: http://aumi.ru --[[User:ISage|ISage]] 17:33, 9 August 2007 (UTC)


Line 44: Line 44:
*FILE size={int8 size}&ed2k={str ed2khash}[&fcode={int4}&acode={int4}]
*FILE size={int8 size}&ed2k={str ed2khash}[&fcode={int4}&acode={int4}]


Is the UDP API supporting other cheksum formats? Would it be too much trouble to implement i.e. md5 checksums?
Is the UDP API supporting other checksum formats? Would it be too much trouble to implement i.e. md5 checksums?
: Needs db support, meaning exp has to grant it. --[[User:Epoximator|Epoximator]] 09:34, 9 July 2007 (UTC)
: Needs database support, meaning exp has to grant it. --[[User:Epoximator|Epoximator]] 09:34, 9 July 2007 (UTC)
:: Uniqueness of all other hashes besides the ed2k hash is not enforced by AniDB. Meaning their use in a FILE lookup might cause some problems in case multiple files with that hash are in the db. [[User:Exp|Exp]] 09:41, 9 July 2007 (UTC)
:: Uniqueness of all other hashes besides the ed2k hash is not enforced by AniDB. Meaning their use in a FILE lookup might cause some problems in case multiple files with that hash are in the db. [[User:Exp|Exp]] 09:41, 9 July 2007 (UTC)


That's odd. How come AOM supports different hash-methods? (I know this may not be asked here) And shouldn't the md5sum be unique, too? I mean uniqueness IS the reason for using hash... Well I guess we can handle everything through ed2k though. [[User:Ainawing|Ainawing]] 15:49, 9 July 2007 (GMT+1)
That's odd. How come AOM supports different hash-methods? (I know this may not be asked here) And shouldn't the md5sum be unique, too? I mean uniqueness IS the reason for using hash... Well I guess we can handle everything through ed2k though. [[User:Ainawing|Ainawing]] 15:49, 9 July 2007 (GMT+1)
: Well, the hashes are likely to be unique. However, this is not enforced by AniDB. Whether two distinct files really have the same hash or whether it is just an input error (i.e. wrong c&p) of the submitting user, the result is the same. You're trying to do a lookup for which you expect exactly one result or nothing at all and you end up with multiple results. And yes, most clients do support other hashing methods and also submit such data to AniDB. However, internally they are all based on ed2k hashes AFAIK. [[User:Exp|Exp]] 08:52, 10 July 2007 (UTC)
: Well, the hashes are likely to be unique. However, this is not enforced by AniDB. Whether two distinct files really have the same hash or whether it is just an input error (i.e. wrong c&p) of the submitting user, the result is the same. You're trying to do a lookup for which you expect exactly one result or nothing at all and you end up with multiple results. And yes, most clients do support other hashing methods and also submit such data to AniDB. However, internally they are all based on ed2k hashes AFAIK. [[User:Exp|Exp]] 08:52, 10 July 2007 (UTC)
:: The real question is: Should we enforce uniqueness? It's not really a problem to implement and shouldn't lead to any problems. We have reports on uniqueness and there are atm 2x2 files with equal md5, which obviously is wrong (if you look at them). Constraints would just mean that DerIdiot doesn't have to go around and fix such entries from time to time. On the other hand, supporting md5 lookup for clients is hardly important and it'll only have a negative impact on performance (although probably unnoticeable). --[[User:Epoximator|Epoximator]] 12:02, 10 July 2007 (UTC)
:: The real question is: Should we enforce uniqueness? It's not really a problem to implement and shouldn't lead to any problems. We have reports on uniqueness and there are ATM 2x2 files with equal md5, which obviously is wrong (if you look at them). Constraints would just mean that DerIdiot doesn't have to go around and fix such entries from time to time. On the other hand, supporting md5 lookup for clients is hardly important and it'll only have a negative impact on performance (although probably unnoticeable). --[[User:Epoximator|Epoximator]] 12:02, 10 July 2007 (UTC)
::: Well, enforcing uniqueness in the db brings a slight performanc overhead with it, but as file additions are a very seldom event, that wouldn't hurt us. We'd have a couple of extra indicies on the file table, which would increase the storage requirements of the db. Though even that wouldn't be all that much. Supporting MD5 hashes on the UDP API might simplyfy the writing of very simple UDP clients, as there are easily available MD5 libraries for every programming language out there. For ed2k/md4 libs you might have to search for a bit. However, I think I wouldn't go as far as to enforce uniqueness for all our hashes (i.e. sha1 and tth). But it might be seriously worth considering enforcing MD5 uniqueness, especially if we might somday drop ed2k hashes. MD5 hashes would offer a nice fallback in such a case. [[User:Exp|Exp]] 08:52, 11 July 2007 (UTC)
::: Well, enforcing uniqueness in the DB brings a slight performance overhead with it, but as file additions are a very seldom event, that wouldn't hurt us. We'd have a couple of extra indices on the file table, which would increase the storage requirements of the DB. Though even that wouldn't be all that much. Supporting MD5 hashes on the UDP API might simplify the writing of very simple UDP clients, as there are easily available MD5 libraries for every programming language out there. For ed2k/md4 libs you might have to search for a bit. However, I think I wouldn't go as far as to enforce uniqueness for all our hashes (i.e. sha1 and tth). But it might be seriously worth considering enforcing MD5 uniqueness, especially if we might some day drop ed2k hashes. MD5 hashes would offer a nice fallback in such a case. [[User:Exp|Exp]] 08:52, 11 July 2007 (UTC)
:::: Agreed. Even better, since hashes are only being used for identification and checksumming, and not authentication, there's no real need for uniqueness other than the 1 in several trillion chance of a collision. The odds get even better with SHA1 or Tiger or RIPEMD. ed2k is not exactly a standard hash, and MD4 is simply not present in some languages, unless you obtain an external library. (For example, I bundle a native Python MD4 with my client, since that library is rare outside of Linux distributions.) [[User:Billessig|Billessig]] 09:07, 26 July 2007 (UTC)
:::: Agreed. Even better, since hashes are only being used for identification and checksumming, and not authentication, there's no real need for uniqueness other than the 1 in several trillion chance of a collision. The odds get even better with SHA1 or Tiger or RIPEMD. ed2k is not exactly a standard hash, and MD4 is simply not present in some languages, unless you obtain an external library. (For example, I bundle a native Python MD4 with my client, since that library is rare outside of Linux distributions.) [[User:Billessig|Billessig]] 09:07, 26 July 2007 (UTC)
Well, how's it looking? There does not seem to be made any progress on this one. Has this been dropped or simply stalled? :-)  [[User:Ainawing|Ainawing]] 11:24, 19 November 2007 (CET)
Well, how's it looking? There does not seem to be made any progress on this one. Has this been dropped or simply stalled? :-)  [[User:Ainawing|Ainawing]] 11:24, 19 November 2007 (CET)
Line 73: Line 73:


'''Notes:'''
'''Notes:'''
* The lid of the added file is written in the reply only in case just one file was added. ''Actualy, if many files were added, there lid's would by useless to client because it wouldn't know the correspondence between lid's and files.''
* The lid of the added file is written in the reply only in case just one file was added. ''Actually, if many files were added, there lid's would by useless to client because it wouldn't know the correspondence between lid's and files.''


== Notify Commands ==
== Notify Commands ==
Line 87: Line 87:


* AES 128bit, '''CBC mode''', PKCS5 padding
* AES 128bit, '''CBC mode''', PKCS5 padding
** 128bit MD5 sum (raw, don't convert to hex) of api encryption key (api pass) is used as 128bit key.
** 128bit MD5 sum (raw, don't convert to hex) of API encryption key (API pass) is used as 128bit key.
** 128bit MD5 sum (raw, don't convert to hex) of {str salt} in the reply of the ENCRYPT command is used as 128bit IV.
** 128bit MD5 sum (raw, don't convert to hex) of {str salt} in the reply of the ENCRYPT command is used as 128bit IV.


Line 97: Line 97:
*** do not automatically disable encryption if the server fails to decrypt a packet, instead discard the packet. Send an unencrypted error reply "XXX DECRYPTION FAILED - PACKET IGNORED".
*** do not automatically disable encryption if the server fails to decrypt a packet, instead discard the packet. Send an unencrypted error reply "XXX DECRYPTION FAILED - PACKET IGNORED".
** otherwise:
** otherwise:
*** if it is an ENCRYPT packet, disable encryption, discard the old key and iv and then process the ENCRYPT packet (thus reenabling encryption with a new key and iv)
*** if it is an ENCRYPT packet, disable encryption, discard the old key and iv and then process the ENCRYPT packet (thus re-enabling encryption with a new key and iv)
*** for any other type of unencrypted packet, discard the packet. Send an unencrypted error reply "XXX ENCRYPTION ENABLED - UNENCRYPTED PACKET IGNORED"
*** for any other type of unencrypted packet, discard the packet. Send an unencrypted error reply "XXX ENCRYPTION ENABLED - UNENCRYPTED PACKET IGNORED"


Line 118: Line 118:
Feature Request: The purpose is for a client I want to make, which will essentially be a more user friendly method of viewing and manipulating the view of a user's list. Sort by various categories and so on. I would like to be able to view and browse a specified user's list just as I can with the web interface
Feature Request: The purpose is for a client I want to make, which will essentially be a more user friendly method of viewing and manipulating the view of a user's list. Sort by various categories and so on. I would like to be able to view and browse a specified user's list just as I can with the web interface


Probably best implemented to behave just like the mylist commands.
Probably best implemented to behave just like the MyList commands.


'''Command String:'''<br>
'''Command String:'''<br>
Line 141: Line 141:


'''Info:'''
'''Info:'''
* The state field provides information about the location and sharing state of a file in mylist.
* The state field provides information about the location and sharing state of a file in MyList.
* If files are added after hashing, a client should specify the state as 1 (on hdd) (if the user doesn't explicitly select something else).
* If files are added after hashing, a client should specify the state as 1 (on hdd) (if the user doesn't explicitly select something else).
* eps is a list of episodes, e.g. "1-12,14-16,T1"
* eps is a list of episodes, e.g. "1-12,14-16,T1"
Line 147: Line 147:
'''States:'''
'''States:'''
   0 - unknown - state is unknown or the user doesn't want to provide this information
   0 - unknown - state is unknown or the user doesn't want to provide this information
   1 - on hdd - the file is stored on hdd (but is not shared)
   1 - on hdd - the file is stored on HDD (but is not shared)
   2 - on cd - the file is stored on cd
   2 - on cd - the file is stored on CD
   3 - deleted - the file has been deleted or is not available for other reasons (i.e. reencoded)
   3 - deleted - the file has been deleted or is not available for other reasons (i.e. re-encoded)


'''Example:'''
'''Example:'''
Line 157: Line 157:


'''Comments:'''<br>
'''Comments:'''<br>
It would probably be ok to add this, but note that it would be a hassle to fetch a complete mylist. I mean, you can already do this with your own list, but it takes a lot of time because of the datagram rate limit. (It would take at least 8h to fetch my own list). And that is usage we don't really want @ udp api. And a command that returns all the data at once will, of course, not be allowed. So, in other words, if you just want to check if a friend has a specific file, or what files per anime (multiple files found), then ok, but ''view and browse a specified user's list just as I can with the web interface'' is not likely to happen/be allowed.--[[User:Epoximator|Epoximator]] 11:29, 4 June 2006 (CEST)
It would probably be ok to add this, but note that it would be a hassle to fetch a complete MyList. I mean, you can already do this with your own list, but it takes a lot of time because of the datagram rate limit. (It would take at least 8h to fetch my own list). And that is usage we don't really want @ UDP API. And a command that returns all the data at once will, of course, not be allowed. So, in other words, if you just want to check if a friend has a specific file, or what files per anime (multiple files found), then OK, but ''view and browse a specified user's list just as I can with the web interface'' is not likely to happen/be allowed.--[[User:Epoximator|Epoximator]] 11:29, 4 June 2006 (CEST)
:What I'm really interested in is maintaining local dbs of 3/4 user lists and having a client automatically update them every 2/3 weeks. We use this to coordinate information within our country regarding who has what =p. But what we're interested in is a way to be able to selectively retrieve the entire list by some index. OFcourse if its slow, then it makes sense to maintain it locally and just look for changes. The other thing I really want is to somehow retrieve with mylist file data (or user list file data), the date at which something was added to the list. (I understand this information exists). The idea here is we can locally query these lists for files added within the last 2/3 months. --[[User:path|path]] 11:53, 25 June 2006 (IST)
:What I'm really interested in is maintaining local dbs of 3/4 user lists and having a client automatically update them every 2/3 weeks. We use this to coordinate information within our country regarding who has what =p. But what we're interested in is a way to be able to selectively retrieve the entire list by some index. Of course if its slow, then it makes sense to maintain it locally and just look for changes. The other thing I really want is to somehow retrieve with MyList file data (or user list file data), the date at which something was added to the list. (I understand this information exists). The idea here is we can locally query these lists for files added within the last 2/3 months. --[[User:path|path]] 11:53, 25 June 2006 (IST)
::That's really not wanted usage for this api. Have you thought about the possibility to use mylist export? You know you can make you own export template and get all the info you want at once? A command that query all changes (lids) within the last 2/3 months (or even weeks) is not going to be added. --[[User:Epoximator|Epoximator]] 12:36, 25 June 2006 (UTC)
::That's really not wanted usage for this API. Have you thought about the possibility to use MyList export? You know you can make you own export template and get all the info you want at once? A command that query all changes (lids) within the last 2/3 months (or even weeks) is not going to be added. --[[User:Epoximator|Epoximator]] 12:36, 25 June 2006 (UTC)




Line 165: Line 165:
=== Web Service API ===
=== Web Service API ===


I think most of the UDP API funcionality could be exposed as a web service. Not only it would  make client programming simpler but could also solve some NAT/Proxy/Firewall UDP problems. This web service would only provide query capabilities, since callbacks (required for notifications) in web services still pose some problems. --[[User:Jassuncao|Jassuncao]] 08:43, 20 September 2006 (UTC)
I think most of the UDP API functionality could be exposed as a web service. Not only it would  make client programming simpler but could also solve some NAT/Proxy/Firewall UDP problems. This web service would only provide query capabilities, since callbacks (required for notifications) in web services still pose some problems. --[[User:Jassuncao|Jassuncao]] 08:43, 20 September 2006 (UTC)


:Web services are for big idiot corps with rooms of servers and no competent programmers. Spend TCP->HTTP->SOAP overhead to save on personnel. --[[User:Rar|Rar]] 21:49, 23 September 2006 (UTC)
:Web services are for big idiot corps with rooms of servers and no competent programmers. Spend TCP->HTTP->SOAP overhead to save on personnel. --[[User:Rar|Rar]] 21:49, 23 September 2006 (UTC)
1,633

edits

MediaWiki spam blocked by CleanTalk.
MediaWiki spam blocked by CleanTalk.