SigServer DEV: Difference between revisions

From AniDB
Jump to navigation Jump to search
mNo edit summary
 
(22 intermediate revisions by 6 users not shown)
Line 1: Line 1:
{{TOCright}}
{{TOCright}}
{{eyecatch|Note|'''This page only lists development resources. If you're interested in using the AniDB Signature Server, visit: [[SigServer]]'''}}
=General=
=General=
this is the place to contribute ideas on a possible future addition of a special signature picture feature to anidb.
This is the place to contribute ideas on a possible future addition of a special signature picture feature to AniDB.


Related forum thread: [http://www.anidb.net/forum/viewtopic.php?t=5413 AniDB Development Forum - SigServer]
Related forum thread: [http://www.anidb.net/forum/viewtopic.php?t=5413 AniDB Development Forum - SigServer]
Line 8: Line 11:


=Vision=
=Vision=
AniDB users can create highly personalized signature images which can be used in forum singatures and which acurately reflect user specific or general anidb data. Some examples are:
AniDB users can create highly personalised signature images which can be used in forum signatures and which accurately reflect user specific or general AniDB data. Some examples are:
* Mylist statistics of the user (Mylist size, latest added, latest watched, total watched, ...)
* MyList statistics of the user (MyList size, latest added, latest watched, total watched, ...)
* Vote statistics general or user based (Top3 animes, my Top3, worst anime ever, ...)
* Vote statistics general or user based (Top3 anime, my Top3, worst anime ever, ...)
* General anidb statistics
* General AniDB statistics
* Group statistics (Latest releases by group X, ...)
* Group statistics (Latest releases by group X, ...)
* Anime statistics (percent aired, percent subbed, latest file added, total users, total groups, total files, total eps, ...)
* Anime statistics (percent aired, percent subbed, latest file added, total users, total groups, total files, total eps, ...)
Line 18: Line 21:
The user would not need any webspace to host such images and would not need to update/regenerate them. All signature images would be directly loaded from an AniDB webserver.
The user would not need any webspace to host such images and would not need to update/regenerate them. All signature images would be directly loaded from an AniDB webserver.


The idea is for the sig server to be the middle entity between AniDB and the user, which allows almost any kind of singature pic to be created, according to the personal preferences of the user. As such a lot of attention should go towards the configuration/customization interface of such a feature. We are aware that this is a hell lot of work, but it's the long term goal. The image generation itself is fairly easy.
The idea is for the sig server to be the middle entity between AniDB and the user, which allows almost any kind of signature pic to be created, according to the personal preferences of the user. As such a lot of attention should go towards the configuration/customisation interface of such a feature. We are aware that this is a hell lot of work, but it's the long term goal. The image generation itself is fairly easy.


Basically AniDB will offer all kinds of stats data and more via xml, much more than is currently on offer. The sig server would transparently get it's data from AniDB and the user will be able to pick all kinds of information bits from the available data and position and format it freely on a background pic which he can upload himself.
Basically AniDB will offer all kinds of stats data and more via XML, much more than is currently on offer. The sig server would transparently get it's data from AniDB and the user will be able to pick all kinds of information bits from the available data and position and format it freely on a background pic which he can upload himself.
Of course many users wouldn't need this degree of flexibility, so there would need to be some common premade default selections for style/info combinations which are most commonly requested. Furthermore background images could be shared between users.
Of course many users wouldn't need this degree of flexibility, so there would need to be some common pre-made default selections for style/info combinations which are most commonly requested. Furthermore background images could be shared between users.


... if a users feels like having a total mess of a sig pic where every single character is done in a different font type, color and size... why not let him do it? ...
... if a users feels like having a total mess of a sig pic where every single character is done in a different font type, colour and size... why not let him do it? ...


Some examples:
Some examples:
Line 38: Line 41:
(Let's call the new server "sig server" for now)
(Let's call the new server "sig server" for now)
* The sig server has it's own user database
* The sig server has it's own user database
* Users can configure the signature pics via a webinterface of the sig server
* Users can configure the signature pics via a web interface of the sig server
* The sig server provides an url which returns the sig pic
* The sig server provides an URL which returns the sig pic
* When the url is requested the sig server will look for a locally cached pic or cached anidb status information for the user and will either return the cached pic, a new pic created with the cached information or will issue an http request to retrieve the needed stats via as XML and then create the pic.
* When the URL is requested the sig server will look for a locally cached pic or cached AniDB status information for the user and will either return the cached pic, a new pic created with the cached information or will issue an http request to retrieve the needed stats via as XML and then create the pic.


It is important to underline at this point that the sig server does never directly interact with the anidb database. It does not have SQL access, it does not store the users anidb password (only their anidb username) and it never modifies any information on anidb.  
It is important to underline at this point that the sig server does never directly interact with the AniDB database. It does not have SQL access, it does not store the users AniDB password (only their AniDB username) and it never modifies any information on AniDB.  


On suggestion for how things could look like (feel free to suggest other ways):
On suggestion for how things could look like (feel free to suggest other ways):
* AniDB will link to the sig server
* AniDB will link to the sig server
* A first time user will visit the sig server and register a new account (username and password can be different from anidb)
* A first time user will visit the sig server and register a new account (username and password can be different from AniDB)
* Once registered the user gets to a configuration page where he has to enter his anidb username (without the anidb password!) and there will be a help text explaining the required profile settings on anidb for this to work (the anidb XML interface will only provide the information if the user selected certain permissions for the guest user)
* Once registered the user gets to a configuration page where he has to enter his AniDB username (without the AniDB password!) and there will be a help text explaining the required profile settings on AniDB for this to work (the AniDB XML interface will only provide the information if the user selected certain permissions for the guest user)
* the user can then either select an image from a selection of publicly available pictures stored on the server or can upload is own picture (for each own picture the user can specify whether it should be made public or not (needs mod review)).
* the user can then either select an image from a selection of publicly available pictures stored on the server or can upload is own picture (for each own picture the user can specify whether it should be made public or not (needs mod review)).
* once a picture is selected the user can select the type of data shown and the formatting from predefined profiles or can define his own profile (profile can be made public)
* once a picture is selected the user can select the type of data shown and the formatting from predefined profiles or can define his own profile (profile can be made public)
Line 54: Line 57:
** font/size/color/location/... for each part
** font/size/color/location/... for each part
** maybe some kind of basic scripting
** maybe some kind of basic scripting
* one user should be able to specify upto N signature picture+profile combinations which can all be active at the same time (maybe 5 ?)
* one user should be able to specify up to N signature picture+profile combinations which can all be active at the same time (maybe 5 ?)
* some nice statistics and a modding interface would be good
* some nice statistics and a modding interface would be good


Line 62: Line 65:
** Just to be clear - You will NOT have access to the AniDB database via SQL. As such you're free to come up with your own database schema for the information you need to store. All AniDB data you need will be available as XML via HTTP GET requests.
** Just to be clear - You will NOT have access to the AniDB database via SQL. As such you're free to come up with your own database schema for the information you need to store. All AniDB data you need will be available as XML via HTTP GET requests.
** The SigServer will require a separate signup and will not make use of the AniDB user account data.
** The SigServer will require a separate signup and will not make use of the AniDB user account data.
* Currently the server only runs PHP4 [http://sig.anidb.info/phptest.php Php Info], but PHP5 could be installed.
* Currently the server only runs PHP4 [http://sig.anidb.info/phptest.php PHP Info], but PHP5 could be installed.


==Data Retrieval==
==Data Retrieval==
The SigServer will receive the required data by making HTTP GET requests to the main anidb server and by getting XML replies.
The SigServer will receive the required data by making HTTP GET requests to the main AniDB server and by getting XML replies.
Here are some examples.
Here are some examples.


CAUTION! - Don't access these urls more than one or twice a day or you will get yourself an automatic 24h ban!
CAUTION! - Don't access these URLs more than once or twice a day or you will get yourself an automatic 24h ban!
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=file http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=file]
:* latest files added to MyList
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=anime http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=anime]
:* latest anime for which files were added to MyList (each anime is listed only once).
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=watched http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=watched]
:* list of files which the user marked as watched. Ordered by the view date.
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=watching http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=watching]
:* anime entries for which the user has watched at least one ep within the last 14 days. Ordered by view date of the last recently viewed file per anime.
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=finishedwatching http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=finishedwatching]
:* anime entries which the user has completely watched within the last 2 months. Ordered by view date of the last recently viewed file per anime.
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=vote http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=vote]
:* most recent votes of a user
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=review http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=review]
:* most recent reviews of a user
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=recs http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userlatest&uid=32&list=recs]
:* most recent recommendations made by a user


* [http://anidb.info/perl-bin/animedb.pl?show=xml&t=userlatest&uid=32&list=file http://anidb.info/perl-bin/animedb.pl?show=xml&t=userlatest&uid=32&list=file]
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userstat&uid=32 http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=userstat&uid=32]
* [http://anidb.info/perl-bin/animedb.pl?show=xml&t=userlatest&uid=32&list=anime http://anidb.info/perl-bin/animedb.pl?show=xml&t=userlatest&uid=32&list=anime]
:* statistics for a user
* [http://anidb.info/perl-bin/animedb.pl?show=xml&t=userstat&uid=32 http://anidb.info/perl-bin/animedb.pl?show=xml&t=userstat&uid=32]
* [http://anidb.info/perl-bin/animedb.pl?show=xml&t=animestat&aid=1 http://anidb.info/perl-bin/animedb.pl?show=xml&t=animestat&aid=1]
* [http://anidb.info/perl-bin/animedb.pl?show=xml&t=animestat&aid=1&uid=32&gid=41 http://anidb.info/perl-bin/animedb.pl?show=xml&t=animestat&aid=1&uid=32&gid=41]
* ... lot's more to be added uppon request


The mylist export documentation may be helpful in understanding the meaning of the different fields in the XML replies: [[Mylist_export_template_guidelines]]
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1 http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1]
:* statistics for an anime
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1&gid=41 http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1&gid=41]
:* statistics for an anime, together with group info
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1&uid=32 http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1&uid=32]
:* statistics for an anime, together with user MyList info
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1&uid=32&gid=41 http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=animestat&aid=1&uid=32&gid=41]
:* statistics for an anime, together with user MyList and group info
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidbstat http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidbstat]
:* global AniDB statistics
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=file http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=file]
:* latest files added to AniDB
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=anime http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=anime]
:* latest animes for which files were added to AniDB (each anime is listed only once).
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=review http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=review]
:* latest reviews added to AniDB
 
* [http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=recs http://anidb.net/perl-bin/animedb.pl?show=xmlsig&t=anidblatest&list=recs]
:* latest recommendations added to AniDB
 
* ... lot's more to be added on request
 
The MyList export documentation may be helpful in understanding the meaning of the different fields in the XML replies: [[MyList export template guidelines]]
 
=== Feature Requests ===
* add requests for additional data here
* Latest votes - for each anime, tmp, episodes, groups, reviews --[[User:Suppy|Suppy]] 18:13, 28 April 2007 (UTC) - partly done [[User:Exp|Exp]] 15:10, 17 November 2007 (CET)
* Top votes - for each anime, tmp, episodes, groups, reviews --[[User:Suppy|Suppy]] 18:13, 28 April 2007 (UTC)
* Your top rated reviews --[[User:Suppy|Suppy]] 00:58, 30 April 2007 (UTC)
 
Old:
* Latest watched files. - antennen - done [[User:Exp|Exp]] 15:10, 17 November 2007 (CET)
* Feed for resolving UserID etc. - antennen - done [[User:Exp|Exp]] 15:10, 17 November 2007 (CET)
* Your latest written reviews --[[User:Suppy|Suppy]] 00:58, 30 April 2007 (UTC) - done [[User:Exp|Exp]] 15:10, 17 November 2007 (CET)


==XML Parsing==
==XML Parsing==
Line 84: Line 147:
To parse the XML, standard php-commands can be used.
To parse the XML, standard php-commands can be used.


A working example for xml.xml with the output array stack():
A working example for <tt>xml.xml</tt> with the output array stack():
{|  width="70%"
{|  width="70%"
|<pre><nowiki>
|<pre><nowiki>
Line 195: Line 258:
</nowiki></pre>
</nowiki></pre>
|}
|}
[[Category:SigServer]]
[[Category:Development]]
[[Category:API]]

Latest revision as of 20:39, 27 April 2009

Note This page only lists development resources. If you're interested in using the AniDB Signature Server, visit: SigServer

General

This is the place to contribute ideas on a possible future addition of a special signature picture feature to AniDB.

Related forum thread: AniDB Development Forum - SigServer

For other areas of active development on AniDB, check: Development

Vision

AniDB users can create highly personalised signature images which can be used in forum signatures and which accurately reflect user specific or general AniDB data. Some examples are:

  • MyList statistics of the user (MyList size, latest added, latest watched, total watched, ...)
  • Vote statistics general or user based (Top3 anime, my Top3, worst anime ever, ...)
  • General AniDB statistics
  • Group statistics (Latest releases by group X, ...)
  • Anime statistics (percent aired, percent subbed, latest file added, total users, total groups, total files, total eps, ...)
  • ...

The user would not need any webspace to host such images and would not need to update/regenerate them. All signature images would be directly loaded from an AniDB webserver.

The idea is for the sig server to be the middle entity between AniDB and the user, which allows almost any kind of signature pic to be created, according to the personal preferences of the user. As such a lot of attention should go towards the configuration/customisation interface of such a feature. We are aware that this is a hell lot of work, but it's the long term goal. The image generation itself is fairly easy.

Basically AniDB will offer all kinds of stats data and more via XML, much more than is currently on offer. The sig server would transparently get it's data from AniDB and the user will be able to pick all kinds of information bits from the available data and position and format it freely on a background pic which he can upload himself. Of course many users wouldn't need this degree of flexibility, so there would need to be some common pre-made default selections for style/info combinations which are most commonly requested. Furthermore background images could be shared between users.

... if a users feels like having a total mess of a sig pic where every single character is done in a different font type, colour and size... why not let him do it? ...

Some examples:

Data

Implementation

General

here a short outline of the way it is supposed to work: (Let's call the new server "sig server" for now)

  • The sig server has it's own user database
  • Users can configure the signature pics via a web interface of the sig server
  • The sig server provides an URL which returns the sig pic
  • When the URL is requested the sig server will look for a locally cached pic or cached AniDB status information for the user and will either return the cached pic, a new pic created with the cached information or will issue an http request to retrieve the needed stats via as XML and then create the pic.

It is important to underline at this point that the sig server does never directly interact with the AniDB database. It does not have SQL access, it does not store the users AniDB password (only their AniDB username) and it never modifies any information on AniDB.

On suggestion for how things could look like (feel free to suggest other ways):

  • AniDB will link to the sig server
  • A first time user will visit the sig server and register a new account (username and password can be different from AniDB)
  • Once registered the user gets to a configuration page where he has to enter his AniDB username (without the AniDB password!) and there will be a help text explaining the required profile settings on AniDB for this to work (the AniDB XML interface will only provide the information if the user selected certain permissions for the guest user)
  • the user can then either select an image from a selection of publicly available pictures stored on the server or can upload is own picture (for each own picture the user can specify whether it should be made public or not (needs mod review)).
  • once a picture is selected the user can select the type of data shown and the formatting from predefined profiles or can define his own profile (profile can be made public)
  • a profile should be very configurable, allowing selections like:
    • information to display
    • font/size/color/location/... for each part
    • maybe some kind of basic scripting
  • one user should be able to specify up to N signature picture+profile combinations which can all be active at the same time (maybe 5 ?)
  • some nice statistics and a modding interface would be good

Hosting

We have a dedicated server ready for hosting this feature. Restrictions:

  • Database related code should be able to work with PostgreSQL AND MySQL
    • Just to be clear - You will NOT have access to the AniDB database via SQL. As such you're free to come up with your own database schema for the information you need to store. All AniDB data you need will be available as XML via HTTP GET requests.
    • The SigServer will require a separate signup and will not make use of the AniDB user account data.
  • Currently the server only runs PHP4 PHP Info, but PHP5 could be installed.

Data Retrieval

The SigServer will receive the required data by making HTTP GET requests to the main AniDB server and by getting XML replies. Here are some examples.

CAUTION! - Don't access these URLs more than once or twice a day or you will get yourself an automatic 24h ban!

  • latest files added to MyList
  • latest anime for which files were added to MyList (each anime is listed only once).
  • list of files which the user marked as watched. Ordered by the view date.
  • anime entries for which the user has watched at least one ep within the last 14 days. Ordered by view date of the last recently viewed file per anime.
  • anime entries which the user has completely watched within the last 2 months. Ordered by view date of the last recently viewed file per anime.
  • most recent votes of a user
  • most recent reviews of a user
  • most recent recommendations made by a user
  • statistics for a user
  • statistics for an anime
  • statistics for an anime, together with group info
  • statistics for an anime, together with user MyList info
  • statistics for an anime, together with user MyList and group info
  • global AniDB statistics
  • latest files added to AniDB
  • latest animes for which files were added to AniDB (each anime is listed only once).
  • latest reviews added to AniDB
  • latest recommendations added to AniDB
  • ... lot's more to be added on request

The MyList export documentation may be helpful in understanding the meaning of the different fields in the XML replies: MyList export template guidelines

Feature Requests

  • add requests for additional data here
  • Latest votes - for each anime, tmp, episodes, groups, reviews --Suppy 18:13, 28 April 2007 (UTC) - partly done Exp 15:10, 17 November 2007 (CET)
  • Top votes - for each anime, tmp, episodes, groups, reviews --Suppy 18:13, 28 April 2007 (UTC)
  • Your top rated reviews --Suppy 00:58, 30 April 2007 (UTC)

Old:

  • Latest watched files. - antennen - done Exp 15:10, 17 November 2007 (CET)
  • Feed for resolving UserID etc. - antennen - done Exp 15:10, 17 November 2007 (CET)
  • Your latest written reviews --Suppy 00:58, 30 April 2007 (UTC) - done Exp 15:10, 17 November 2007 (CET)

XML Parsing

To use the XML replies from the server, they have to be parsed and probably cached.

To parse the XML, standard php-commands can be used.

A working example for xml.xml with the output array stack():

//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
//Load and parse XML
//~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

$file = "xml.xml";
$stack = array();

// start_element_handler ( resource parser, string name, array attribs )
function startElement($parser, $name, $attribs)
{
   global $stack;
   $tag=array("name"=>$name,"attribs"=>$attribs);
   array_push($stack,$tag);
}

// end_element_handler ( resource parser, string name )
function endElement($parser, $name)
{
   global $stack;
   $stack[count($stack)-2]['children'][] = $stack[count($stack)-1];
   array_pop($stack);
}

// handler ( resource parser, string data )
function characterData($parser, $data)
{
   global $stack,$i;
  
  
   if(trim($data))
   { 
             $stack[count($stack)-1]['data'] .= $data; 
   }
}
// handler ( resource parser, string target, string data )

function defaultHandler($parser, $data)
{
  
}

function externalEntityRefHandler($parser, $openEntityNames, $base, $systemId,
                                 $publicId) {
   if ($systemId) {
       if (!list($parser, $fp) = new_xml_parser($systemId)) {
           printf("Could not open entity %s at %s\n", $openEntityNames,
                   $systemId);
           return false;
       }
       while ($data = fread($fp, 4096)) {
           if (!xml_parse($parser, $data, feof($fp))) {
               printf("XML error: %s at line %d while parsing entity %s\n",
                       xml_error_string(xml_get_error_code($parser)),
                       xml_get_current_line_number($parser), $openEntityNames);
               xml_parser_free($parser);
               return false;
           }
       }
       xml_parser_free($parser);
       return true;
   }
   return false;
}

function new_xml_parser($file)
{
   global $parser_file;
   $xml_parser = xml_parser_create();
  
   xml_parser_set_option($xml_parser, XML_OPTION_CASE_FOLDING, 0);
  
   xml_set_processing_instruction_handler($xml_parser, "PIHandler");
  
   xml_set_element_handler($xml_parser, "startElement", "endElement");
  
   xml_set_character_data_handler($xml_parser, "characterData");
  
   xml_set_default_handler($xml_parser, "defaultHandler");
  
   xml_set_external_entity_ref_handler($xml_parser, "externalEntityRefHandler");
 
   if (!($fp = @fopen($file, "r"))) {
       return false;
   }
   if (!is_array($parser_file)) {
       settype($parser_file, "array");
   }
   $parser_file[$xml_parser] = $file;
   return array($xml_parser, $fp);
}

if (!(list($xml_parser, $fp) = new_xml_parser($file))) {
   die("could not open XML input");
}

//ERROR
while ($data = fread($fp, 4096)) {
   if (!xml_parse($xml_parser, $data, feof($fp))) {
       die(sprintf("XML error: %s at line %d\n",
                   xml_error_string(xml_get_error_code($xml_parser)),
                   xml_get_current_line_number($xml_parser)));
   }
}
//END
xml_parser_free($xml_parser);