1,633
edits
Maintenance DEV: Difference between revisions
m
no edit summary
mNo edit summary |
mNo edit summary |
||
| Line 63: | Line 63: | ||
The data collected is stored inside of an in-memory Perl hash and any required updates are written back to the database at the end of each chunk in one transaction. | The data collected is stored inside of an in-memory Perl hash and any required updates are written back to the database at the end of each chunk in one transaction. | ||
This leads to one big, monolithic cronjob which creates a lot of database, memory and | This leads to one big, monolithic cronjob which creates a lot of database, memory and CPU load. | ||
Current Runtimes: (13.05.2007) | Current Runtimes: (13.05.2007) | ||
* Anime Stats | * Anime Stats | ||
** 1525 seconds (25 minutes) for <5210 | ** 1525 seconds (25 minutes) for <5210 anime | ||
* Group Stats | * Group Stats | ||
| Line 80: | Line 80: | ||
* Total: 7602 seconds (127 minutes) | * Total: 7602 seconds (127 minutes) | ||
The process is mostly limited by the database (the script uses about 900 seconds of | The process is mostly limited by the database (the script uses about 900 seconds of CPU time). | ||
The key issue here is that these numbers rise all the time. In the early days we've run that script multiple times a day, then once a day, now 3 times a week. If things continue as they are now, we'll reach a point where we can't run it at all any more the way it works right now. | The key issue here is that these numbers rise all the time. In the early days we've run that script multiple times a day, then once a day, now 3 times a week. If things continue as they are now, we'll reach a point where we can't run it at all any more the way it works right now. | ||