This winter I’m part of Google Summer of Code: I’m working on improving the GNOME website. You can check all of my tasks here.
It’s been a long week working with BuddyPress and the tools around it. As I wrote in some previous posts, my 3rd task is to create a community system that integrates the people involved in GNOME and shows what is being done in the project in real time.
I’m using BuddyPress — a full featured WordPress-based social network system that supports members, activities and groups — to handle all the things together.
Activities and profiles
To do so, I’m making the activities timeline of each user be a big stream of things done in GNOME services. For example, every time I commit something to a GNOME git repository, an item will be added to my profile telling about this commit. The same will happen with bugs, translations etc.
Each user will have a profile with the activities done in the project, some bio text with fields like “where I live” and “where I am right now”, his/her friends and groups.
Keeping the geolocation of each user will allow us to know if lapo is real where in the world are the GNOME participants, instead of using some archaic method. Also, we aim to have in the home page a featured map with balloons popping up in real time showing what is happening in the community right now.
Groups
Once the main GNOME.org website will be available in several languages, there is no point keeping subdomains like br.gnome.org with outdated translated content. With groups in the Community website, users will be able to interact each other and post news.
What’s the status of this?
Since last week I’ve been working hard on planning the features of the Community website, along with the BuddyPress theme. The theme itself is keeping me busier than I though, and I’m going to spend more days on it until I get something decent.
I also spent a lot of time studying BuddyPress and some GNOME systems like BugZilla and Damned Lies in order to know how to get fetch generated data.
How to get activity data
In order to fill the profiles with the activities done in GNOME systems, we’ll need specific code for each system.
For BugZilla, there is XMLRPC communication support. I need to study it a bit deeper, but if I’m not wrong it’s not hard to get all the recent actions so I can execute a script getting it regularly.
For Damned Lies, I can get a feed of actions (like this), but only if I specify a language or a module. Hacking it to provide a global feed of actions (and informing the user email in the feed) would solve the case (Stéphane Raimbault, I’m looking at you 
).
For git, I can use server-side post-receive hook or parse cgit feeds (like this). Still need to study this.
There are some not-very-clear parts, like how to handle received activity of users without registered accounts to the Community website. Maybe we should just ignore it .
⁂
For this week I’m planning to have something really done regarding the theme and get the geolocation support for users functional.
I’d love to hear some feedback regarding this. Thanks!
16 Comments
> “For BugZilla, there is XMLRPC communication support. I need to study it a bit deeper, but if I’m not wrong it’s not hard to get all the recent actions so I can execute a script getting it regularly.”
But then that means that nothing will happen for X seconds (the refresh interval) and suddenly a lot of popups will appear on the map.
Depending on the volume of activity, it might be pretty ugly.
And using a very small value of X is not very friendly with the infrastructure resources.
Same goes for Damned Lies feeds.
For Bugzilla at least, you could have an email address CC-ed to each and every event, and a bot that acts on email reception to push the notification to the community website? (I don’t know anything about Damned Lies so I won’t suggest it)
The ideal for real real-time notifications would be to use a messaging system like AMQP, but then that would be quite a lot of work to integrate it withing Bugzilla, Damned Lies, WordPress,… :-/
Hey bochecha.
With a simple algorithm we can keep catching each X seconds a list of the last 5 activities, for example. If there isn’t new activity yet, we get the 5 previous activities and so on. This would avoid having empty intervals showing no popups.
It’s hard to be real time when you’re dealing with many sources of informations (and those sources don’t provide structure to get the data instantly). Having delayed real time is not that bad
.
> “There are some not-very-clear parts, like how to handle received activity of users without registered accounts to the Community website. Maybe we should just ignore it.”
I suppose that the popup notifications contain a link to go see the user’s profile?
In that case, notifications not associated with registered accounts could simply say that “Someone” did this action, and clicking on the link would lead to a page that says:
“If you are this person and want to be identified for your activities, please open an account in the community website.”
The email provided during registration would be checked against the email used for the activity.
If they match, past and future activity gets associated.
If they don’t, that’s ok, it means a new member had the idea to register after clicking on an unidentified contribution.
Look at me there with my big Mexican hat!
This seems pretty awesome! Great post and keep up the good work!
How accurate we would like the data? After every release or whenever a post is slashdotted, we could suffer a problem on bugzilla. Or even with a greedy crawler.
Getting the data twice a day, every hour, every other hour, etc. should give a good idea of what it is going on. Those data could be collected and server by a different machine or some light process (reading the summary, rather than querying the database).
Hey Germán! The community system have its own database table with all the summary done by the users. This way we only need to request from bugzilla the most recent changes every once in a while, which is as light as displaying a list page in bugzilla itself.
The number of things done in the home page would be based in the community database, and not requested every time from the gnome systems
.
maravilhoso ver um brasileiro fazendo um trabalho tao bom pra um projeto tao maravilhoso
Nice work Vinicius!!
Having an async crawler to those systems (bugzilla, damned-lies, git…) that fills a database maybe is the best.
Then the website just have to query that database.
Cheers,
Thank you, Gil!
Yeah, that’s how the activity table works in BuddyPress. We’ll keep a copy of each action in community system so we just have to query that database
See you at Desktop Summit?!
Awesome as usual, can’t wait to play with it
This sounds great
FWIW, damned-lies already keeps entire GNOME GIT check-out (afaik, and if that hasn’t changed in the last few years) and it keeps it updated with every commit: perhaps that could be turned into a service which would be useful for both getting git commit info and damned-lies themselves, so commit notifications are not duplicated.
FWIW, damned-lies needs this to be able to regenerate a POT file and then compare it with the previous version, but since it’s already doing it, perhaps it could be put to even more use — and you wouldn’t be duplicating the effort thus sparing some cpu cycles
Oh, and btw, I can’t see no reason not to update damned-lies to push the data to buddypress when needed.
Anyway, did I mention that all this is _very_ cool?
This looks pretty impressive. Wondering whether you’ve also checked out Shaun’s Blip for the git activity part?
How about blip for all your data crawling needs?
https://live.gnome.org/Blip
I only want to say that all this work is awesome !!!!
Pretty!
One Trackback
[...] Vinicius Depizzol Just another GNOME weblog Skip to content « GSoC Weekly Update #8 [...]