AniDB:AJAX development

Page dedicated to AJAX Development.

Background

Some copy/paste from mail's from sphere.

Mail 1

hi

I've seen the basic stuff of what's going on. I've also tried to tweak the xml specification a little to single out those that are general (to all of anime community) and those that are anidb specific (I use a namespace idea so that anidb stuff like ids, ratings etc are clearly marked).

Also tried throwing in a more general resource tag so that anything can be referenced there. (one example is the anime's main image, any weblinks to external databases, official sites, additional links to images and videos etc)

As I was reading up stuff, it occured to me that if we had some ajax-ready xml api, it could be far more useful than just for ajax. Using XSLT, what was traditionally done on server (transforming data to html) can be pushed to the client. If the data is carefully segmented into largely static ones (like basic anime info that doesn't update a lot) and more dynamic ones (like episodes and files), there is potential gain if inter-server caching can be exploited (since the xsl does not change at all, and the xmls can be chopped up such that some parts do not need to be updated that often and can be locally/isp cached).

The only shortcoming is that xsl is a pure declarative language. So to achive some dynamic formatting (e.g. say if you want every odd row in a table to have different style), may need runtime javascript.

There are some quirks though... e.g. I can't see the generated html source anymore. when I look at source, I see the xml only. dunno whether that's good or bad atm. [Rar-You're missing the fact the transform doesn't have to run in the browser. libxslt has py bindings, try that. Or just CTRL+A/View Selection Source in foxy.]

one nice thing about this is that the AJAX parts can be built directly on the xml so one common data format may serve the various clients (raw html, xslt, full ajax web app (see the javeline demo link I placed in the readme file))

anyway, attached a sample of what I was trying out. it consists of only a xsl and xml file. Just open the xml file in your browser and the xsl will be automatically applied. If you think this is interesting enough to the rest of the mods (regardless of whether it needs to be interested, I just thought that if they did not know yet, at least they should know that this is possible?)

btw, from the looks of things and the political reason you mentioned for not supporting ajax, I really don't know where to go start or where to go from here. If the general mood is to stick to what we have, does that mean that you and I are just working towards a closed door?

Mail 2

last version doesn't have the language icons, but I made it lookup a publicly available language file (http://schmidt.devlib.org/data/language-names.xml) for the language description translation. took me a while to get used to declarative approach but this version has the nice lang icons we get from AniDB's default styles.

Mail 3

sorry for sending you three mails in a row. this is the final update for some time to come.

following update: --standardized some of the tags to allow dynamic searching.. e.g. when main title is needed and not found, it will grab any title that's available. If maiun image not found it will take any image.

--more proper title management. now titles can be given in any manner as long as it defines a language and the cdata. the xslt will automatically find all the unique values, collect all the languages and display them in one row per unique official title.

--fixed the odd-even row formatting. now the formatting should be correct regardless of how many rows are in the info.

--episode list (no files yet) added. sorted on type and episode number. Episode title is chosen by <defaultlang>...</defaultlang> in the config part of xml. if the language is not found, it will take any language. If no title is found, it will display "unknown title"

--pasted back in the menu (guest access).

--all the links on page is based on anidb.net, so any relative link still link correctly back to anidb site

--added another test data (a235.xml eh.. chosen for the large number of official languages)

I think I will take a break until there is feedback. xslt is just a middle ground, but it is probably the basis for future ajax development, as it totally separates the xml's dependency on formatting. If you are still busy, perhaps I can bring this to the forum by creating an ajax development thread?

Forum

this is the thread that originated it all.

http://forum.anidb.net/viewtopic.php?t=5653

Notes

I'm missing the rar's from the mails, i can't upload stuff now, will upload them later. fahrenheit

Some more updates

I've done just a little more but am still holding onto it. Basically, the language codes are now decoded properly using xml's from ISO sources. Whatever the language code is, be it 2 letters/3 letters etc, and whatever locale, as long as it follows the xx, xx_XX or xx-XX format, it will get decoded into language (+Locale if available). I found that the xml file i used earlier only contains language, so it cannot decode langcodes like pt_BR : Portuguese (Brazil) properly. Now I linked in codes from ISO-639 (language) and ISO-3166 (country) so theoretically, it can decode anything that have been seen before.

To reach out to all people, I was dreaming up these channels (just for discussion):

HTML

Needed to support ancient browsers. (including some safari and opera that doesn't do xslt).

• Ups: Simplest to read. No magic there as server does everything.

• Downs: Server does everything, which means that server has to dynamically compose every single page (due to personal settings) which I'd been informed making it difficult to exploit remote http caching.

[Fahrenheit - unfortunately it's true that we have some uncommon browsers connecting that could need html layout, exp can list some examples]

XSLT

• Ups: Server throws out the xml (no html at all), which is also used for ajax. so theoretically, when we have ajax, xslt comes for free. the style sheet is static. meaning can be cached etc. The xml file is also largely static (no dynamism due to user settings. It only needs to change when anime data actually changes.) Also allows splitting of data into domain. e.g. the same basic anime xml used by all. files and member only data can be sent in another separate xml to be merged on client's end. XSLT is available on almost all modern browsers.

• Downs: There are still some browsers that don't do xslt. [Rar-Name one?]

[fahrenheit - I think rar tried something in these lines before]

[sphere - this is a link to a simple list of browsers that do support xslt natively, [1]. One thing to add is that the support here I'm talking about is NATIVE support. Even on a browser with no NATIVE support, though it is not possible to open xml natively and have the stylesheet kick it, it is possible to do XSLT within the AJAX environment through javascript libraries, by manually loading the xml and xsl and merging them using the provided api.]

AJAX

This is supposedly the real workhorse. Unlike opening a XSLT-transformed XML (which does a one time translation), an AJAX client will be a constant webpage that dynamically updates its parts on receipt of new data. E.g. if the user opens another anime, the ajax page will just request the required xmls and update itself. The modern pattern is to implement XSLT/JST into it's components. e.g. say I have a div set to contain anime episodes data. After retrieving the xml, instead of doing the traditional parse by javascript and manually add in stuff, the modern way is to declare a template before hand (potentially stored statically as a separate file) and use a general translater engine to merge the xml/xsl. makes client development a breeze, not to mention achieving component reuse (the xsl can be used in other pages too!)

• Ups: Rich web experience. due to partitioning of xml data, potentially a large portion can be more static and cacheable. meaning server only have to serve the differences. E.g. the server may serve the same anime and episode info to all people, but may send a per-user xml which contains settings, preferences and mylist stuff. Lighten server load even further? potentially allow even more user customization like the skinning system (server tells client which skin file you use, which can be set by you), coz now, you can change the entire layout of the page by setting a different xslt/jst file sets??

[fahrenheit - you can change the CSS stylesheet on the profile options]

[sphere - CSS stylesheet can only style the fixed DOM, with xslt/jst, the same xml can genereate totally different DOMs (which can be further style by individual CSS). But this remains just an option atm, since we're still emulating the original anime and it seems good enough that there seems to be little suggestions for some changes that are so radical that CSS alone is insufficient :)]

• Downs: requires javascript. heavier on client end (but better client than server, imo). Have to be made more robust coz there may be higher chances of computer amateurs messing stuff up on their client end (as compared to server-spoon-fed html)

[fahrenheit - for the first part it can be impractical for large animes, for the second, can happen with server-spoon-fed html, we had some greasemonkey scrips before, you can mess with anything that has a dom tree]

[sphere - for the first, I'll like to think for data of anidb's size, it is still mostly ok as we at most deal with one anime at a time, but this is groundless until we can do some testing]

AJAX ++ (javeline)

This is the future. And it is already here. I've already highlighted the link in my mail to Fahrenheit (www.ajax.org). It is a complete javascript-only, cross-browser development than can do most things that your typical desktop app can do.... windows, lists, trees. you name it, it has it. What makes it impressive is the ease with which it can be used. cross-browser means that no need to deal with differences in browsers directly. It really takes a lot off the mind of the developer in tinkering with javascript hacks to makes things work and allow more energy to be channeled into the important parts : Design. What's more is that the platform is being developed/maintained continuously, so we get "upgrades" automatically when time passes without having to sink in extra manpower to keep things shiny.

• Ups: everything I've mentioned above. did I mention cross-browser?

• Downs: supposedly even more demanding on browser. But from my testings on a typical 1.8 GHz AMD (single-cpu) using firefox 2, it seems to be still very light on cpu.

[Fahrenheit - It still seems to be a bit incomplete, looks nice, but i don't know if this could handle large amounts of data. Still the framework concept has already been bashed a couple of times on to my head, including but now limited to moochikit]
[sphere - I'm not sure about load tolerance too. This has to be tested if we ever moved there. What attracts me here isn't just the set of features (for which there are many libraries which can do very sophisticated stuff). Rather, it is the ease and intuitive way whereby a 'template' is declared. And these templates are potentially reusable by other people. e.g. From a single AJAX DOM, I could have one template that generates the basic anime info, another that generates the episode list. These two templates can be used on other pages as well to provide the exact view by linking in the template content which can be stored statically somewhere. The typical page designer just do things like 1. create a div, 2. load an xml and a compatibe xsl, 3. call some javascript function to render the page. Similar to styles, these template files can be customized and 'shared' between users.]

Multilingual support

[Sphere] Added this as afterthought. I tried fooling around with the basic anime info xslt and managed to add in translation time multilingual support. basically I created a very simple string table in another xml that specifies a language and a list of key-value pairs that denotes a key in some default language (english in this case) and a translated value.

<presentation> <animeinfo> <stringtable lang="xx"> <string id="1"><key>main</key> <value>niam</value></string> <string><key>search</key> <value>hcraes</value></string> <string><key>calander</key> <value>rednalac</value></string> </stringtable> </animeinfo> </presentation>

Comments, propositions and whatever else

Write comments here.