How to program without losing too much hair

Getting Thunderbird to work with distribution.ini

2011-10-18T17:12:00.016-04:00

The bug I'm working on is the ability to customize vanilla Thunderbird with a group of settings (through distribution.ini). For those who do not know, Thunderbird is Mozilla's open source mail client. (Mozilla = makers of Firefox). I'm far from done. This entry is a culmination of what I have learned so far.

Checkout the repo

I am working with Fedora, so I use yum. I followed the build and configure instructions from here. In a nutshell, here are the steps to cloning the comm-central (Thunderbird) repo:

1) yum install mercurial - to get Mercurial, a version control tool.

2) Create a file called ".hgrc" in your home directory and put these in it:

[ui]
username=Firstname Lastname
merge=internal:merge

[diff]
git=1
showfunc=1
unified=8

[defaults]
commit=-v

Note: Obviously, replace Firstname, Lastname, and my.email... with your information.

3) Make sure you're in the directory that you want to put the repo in (for example, your home dir) then checkout the repo:
hg clone http://hg.mozilla.org/comm-central/ src-thunderbird

So now you should have a "src-thunderbird" folder in your current directory.

Building Thunderbird

1) Go to your src-thunderbird directory (cd src-thunderbird).

2) Create a .mozconfig file. (I use vi, so vi .mozconfig). This file will contain your build configurations for Thunderbird. The contents of mine are:

mk_add_options MOZ_OBJDIR=@TOPSRCDIR@/obj-tbdebug

ac_add_options --enable-application=mail

ac_add_options --enable-debug

ac_add_options --disable-optimize

"obj-tbdebug" is the directory where all my object files will end up (you can change what you call yours). I will refer to this as objdir, from now on. The second line says, build me Thunderbird. The third line is because I need debugging printouts. The fourth line I just threw in for good measure. You can read more about configuration options here.

3) Build Thunderbird! "make -f client.mk build" (while still in src-thunderbird). This step will take a while (more than a half hour in my case).

4) Once the build is done, run Thunderbird with ./objdir/mozilla/dist/bin/thunderbird. Since we configured this build with the debug option, you will see a lot of printouts at your terminal.

Updating the repository

Make sure you're in the src-thunderbird directory. On the terminal, execute python client.py checkout. Re-build again.

Modifying the source files

The source files are everything outside your src-thunderbird file. Everything in it are your object files. When you modify your source files, you need to re-create your object files. You can run make -f client.mk build again (in your src-thunderbird dir), and it won't take as long as the first time.

If you changed files in only one directory, you can re-make only that directory. For example, if you changed files only in ~/src-thunderbird/mail/components, run make -C objdir/mail/components.

Getting distribution.ini working

1) This is a sample distribution.ini. This file must be placed in objdir/mozilla/dist/bin/distribution.

2) Add "distribution.js \" to the EXTRA_JS_MODULES section of Makefile.in (which resides in ~/src-thunderbird/mail/base/modules).

3) distribution.js should be in ~/src-thunderbird/mail/base/modules. I copied and modified the Firefox version. (Code referring to bookmarks and livemarks were ripped out.) I also changed the name of the object it exported to something like TBDistCustomizer.

(Note: JS modules have EXPORTED_SYMBOLS at the start of the file. This is as opposed to components files (which are also JS), which do not.)

4) cd to objdir/mail/base/modules and type "make".

5) Modify mailGlue.js (in ~/src-thunderbird/mail/components) so that:

(a) It would import distribution.js (Components.utils.import("resource:///modules/distribution.js");); and

(b) it would create an instance of TBDistCustomizer. This intance can now be used to call the code in distribution.js that actually changes prefs (applyPrefDefaults).

Hitting the brick wall

So now I've come to the point where preferences set in distribution.ini are being set in Thunderbird, but there is a catch: some prefs are set properly, while others are duplicated. For example, mail.phishing.detection.enabled is set correctly, but something like network.cookie.lifetimePolicy instead becomes duplicated, with one version having the default and the other having the value that was set in distribution.ini.

I am still trying to figure out why this is so...

Climbing the brick wall

The problem was that distribution.ini doesn't like spaces around the "=" sign. network.cookie.lifetimePolicy = 1 should be network.cookie.lifetimePolicy=1.

Thanks to...

The MailDev team at Mozilla! In particular Standard8 and sid0 for helping me get started.

Mozilla XPCom

2011-06-15T12:57:00.002-04:00

Here is the link to a list of ALL the XPCOM contract ID's/components and interfaces are: http://www.oxymoronical.com/experiments/xpcomref/

I am currently working on creating an addon that will always take settings from Internet Explorer and set the same prefs for Firefox. Still in the research stages, tho.

SHUTDOWN ABORT

2011-05-24T21:19:00.002-04:00

I feel powerful executing the above command as a sysadmin 3=) While practicing on my own database for DBA625, I happened to close a session after thinking that zenit has once again frozen. As it turns out, closing session doesn't necessarily end all processes in my database. When I tried logging on again, I got this:


SQL> connect sys as sysdba
Enter password:
Connected to an idle instance.
SQL> startup
ORA-01012: not logged on
SQL> startup mount;
ORA-01012: not logged on

After some googling and deciding that I do not want to mess with resetting memory sizes, I came across "SHUTDOWN ABORT;" and it fixed all my problems.


SQL> SHUTDOWN ABORT
ORACLE instance shut down.
SQL> STARTUP
ORACLE instance started.

Total System Global Area  631914496 bytes
Fixed Size                  1338364 bytes
Variable Size             440402948 bytes
Database Buffers          184549376 bytes
Redo Buffers                5623808 bytes
Database mounted.
Database opened.

Lessons in creating Firefox Enterprise

2011-04-07T17:04:00.009-04:00

Firefox Enterprise

For the past 6 months, I have been at CDOT as Mike Hoye's cronie, and along with scott and annasob, I have been working towards creating an enterprise version of Mozilla's Firefox. Bespoke IO, mhoye's company, will be selling the software that lets administrators customize Firefox and produces the MSI that makes deployment across a network easier. It will also be offering Sync services. If you are not familiar with that, Sync lets you store your browsing history in a server--your own or Mozilla's--so that you can you can access your browsing history, passwords, and bookmarks across several different devices. I will leave the elevator pitch for the boss.

As far as I know, an enterprise version of Firefox conflicts with Mozilla's philosophies about freedom on the web. (To illustrate what "freedom on the web" means, an example: if a user wants to watch a video on YouTube, s/he should just be able to click on a link and the video will play instead of being told that s/he needs a plugin.) That is why Mozilla has not produced the product themselves, and an opportunity for my co-op position was available.

Since Mozilla is open source, it was a matter of taking the existing Build Your Own Browser code and customizing it (with a lot of guidance from kev and lorchard!). The first screenshot is what our BYOB was near the beginning of the project. The first thing I added was the Homepage tab. The second screenshot is what the latest version looks like. The administrator can now choose to lock some things, like the homepage, proxy settings, and which updates are allowed.

Wading through code

It was a mess :s As mhoye put it, he threw me under a bus. Although the framework BYOB was built on was documented (Kohana), for me, there was no way to actually understand what was happening in BYOB unless I did a million printouts and walkthroughs. I realized only much later that the code was exactly what they taught Seneca students in the SYS courses. You have your views, controllers, and models, and then the database.

mhoye had me started on letting an admin choose what the homepage was. Where do I begin? My coding bff since then has been the Multi-file Search option in Textwrangler. I searched for all instances of General, then Locales, then Collections in the hopes that I could trace what happened whenever these words popped up. The problem was that General was useless and Locales was coded differently from little prefs like homepage and proxy settings.

Luckily, kev and lorchard were kind enough to point me in the right direction. But then, it was like giving me a clue as opposed to telling me exactly what to do, which was better in the end. Teach a man how to fish and all that.

Since then, I have gotten to know BYOB really well, but I still learn something everyday, whether it's coding in PHP or bits of Firefox itself. (I dabble in Python every other month, too, but I avoid that part of the code as much as possible.) So far, I can lock down the homepage, proxy settings, and updates. End users cannot change them. Administrators can also add certificates and specify what server they want to use with Sync (if they want it to be used at all).

Challenges

Mozilla's goal is to give the user the best web experience possible; mine is to cramp the user's style just a little bit.

The last great roadblock I had was both a coding problem and a Firefox issue: how do I get BYOB to upload certificates and then add them to the list of certificates in Firefox right from the beginning?

The frustrating thing about coding with Kohana was that I didn't grasp the idea of routes right away. There is a special file in the framework that says, if your url is like this: http://myapp.com/fnName/randomText/moreRandom/, then you should be accessing the first file with function fnName in it (there could be more than one!), and pass it the "*[Rr]andom*" arguments.

I also came up with what I believe to be a rather inelegant solution to adding certificates to Firefox... only to find out that unless I find a way to make the certificate window read-only/disappear, a user can delete any certificate in it. That's problem # 2.

Problem # 1 is bookmarks. Adding bookmarks to a Firefox build was something that Moz's BYOB already did, but how do I lock those bookmarks so that users can't delete them? I know of places.sqlite, bookmarks.html,and the bookmarkbackups folder... But how do I fiddle with those three so that my bookmarks can't be deleted? Right now, I can create a file with json-formatted bookmarks data in it and name it so that Firefox will always load those bookmarks when the browser starts. But that prevents any bookmarks the user will add from loading. Not very user-friendly.

Problem # 3 is preventing the user from deleting addons. Right now, addons are locked so that a user can disable them, but not delete them. From the reading I've done, there is a file called userChrome.css that I can write myself; it prevents the addon manager from appearing at all. I could insert that file into the depths of the Firefox application directory. Is there a better way?

What's next?

Sometime in the summer semester, I believe mhoye is looking for beta testers, so a deadline is tentatively set. My goal for the next week is to leave some good documentation (so that I don't curse myself when I come back from the semester break) and get the bookmarks locked. Still not sure what to do... If you have any answers, please share :D

Game programming with open source tools

2010-10-07T15:32:00.011-04:00

Remember trying to create games with Flash? No? Neither do I. I couldn't be bothered to pay for the tools I needed. But that's okay because it's now possible to code games for the web using free tools like Processing.js and C3DL. You don't have to pay $$$ to be the game programmer you want to be. You just need a bit of patience and dedication.

Level Up: An OpenWeb Game Jam is a tutorial/hack session happening right before the Free Software and Open Source Symposium (FSOSS) 2010 on October 27 at the Seneca@York. Think of it as a one-day sprint of learning new languages. In the morning, you attend the tutorials and the in the afternoon you get to try out what you've learned. Yes, you get to code your own game!

When you really think about it, why learn a new language by yourself when you can have fun, socialize (network, maybe?), and learn with others?

* * *
Some html5 games that you can while away time with:
Chain Reaction - Advanced - http://yvoschaap.com/chainrxnadvanced/
HTML5 Games - http://html5games.com/

Getting tweets to fade onto the screen

2010-09-28T18:29:00.035-04:00

Over the past two weeks, I got to know JSON and the Twitter API well enough to write a fun little function that lets tweets fade into the screen. It is my contribution to the demo that Scott and Anna are presenting at the Open Video Conference. (It was modified in the final version.)

Things to get out of the way:

1) "Closure" in Javascript. According to two explanations (Patrick Hunlock and Javascript Kit), a closure is a variable that continues to exist outside of the function that created it.

2) You can assign a function to a variable! Eg.

var doSomething = function() {
alert("You are perusing Donna's blog");
/* And whatever else you want to do in this function */
}

What it is useful for, as explained in permadi.com, is adding functions as a property of an object (see Example D2A in the webpage).

JSON

To get started, I had to look up what a JSON object is and how to get a JSON object from websites like Twitter. JSON is javascript object notation. All it does is contain data. At its most basic form, a JSON object has a key-value pair (like ({"name" : "donna oberes"}) ); other times, it can be an array of key-value pairs, like this:
("students" : {("name" : "donna oberes", "program" : "cpa"), ("name" : "john doe", "program" : "cns")}). .

Twitter API

A part of the Twitter API describes the link that will bring up a JSON object containing tweets and other information related to it. For example, http://twitter.com/status/user_timeline/daoberes.json?count=15&callback=? will bring up a JSON object containing all the tweets that a person has made and all related information, like the date and the tweet ID. Something like http://search.twitter.com/search.json?q=%23happy&callback=? will bring up all the tweets with the hashtag "#happy" and other info such as the tweeter's ID, the tweet itself, the date, the user's avatar, etc. If you click on the last link (Google Chrome displays the data on the browser), you'll notice that the object has a key called "results" and that its value is an array of objects and other key-value pairs (look at the bottom!). To get it formatted, go to jsbeautifier.org.

jQuery

How did I put all of this together? Through good ol' Javascript and jQuery. According to the website, "jQuery is a fast and concise JavaScript Library that simplifies HTML document traversing, event handling, animating, and Ajax interactions for rapid web development." It did make my life a lot easier.

The functions that I used from the library are .getJSON (), .css (the 2nd overloaded function), $.fadeIn(), and .fadeTo.

.getJSON returns the results of the query in JSON format and then lets you specify exactly what you want done with the returned data with the optional callback parameter. This is what I did:


$.getJSON('http://search.twitter.com/search.json?q=%23' + searchTerm + '&callback=?',
null, function(data) {
datalength = 4;

/* Store tweets and twitter user images in a JSON object */
tweets = []; //declare array of tweets;

//Declare a JSON object
jsonTweets = [];

//Iterate through JSON object and push Twitter pictures and tweets
// onto another JSON object
  for (i = 0; i < datalength; i++) {
    if (data.results[i].text) {
      jsonTweets.push({ "tweetPic" : data.results[i].profile_image_url,
                        "tweet" : data.results[i].text,
                        "twitterId" : data.results[i].from_user,
                        "id" : data.results[i].id });
    }
  }
  nextTweetOrNone();
});

How the function worked

After asking the user to enter a hashtag they would like to print to the screen, the function calls the .getJSON function ( $.getJSON('http://search.twitter.com/search.json?q=%23' + tag + '&callback=?', null, function(data) { ... }); ) and processes the JSON object as a whole and takes out of it anything I would need later on.

nextTweetOrNone() is going to check if the array I just created still has any elements in it. I am going to start popping elements out of it in the next function, and then I will call nextTweetOrNone() again to keep checking if there are still elements I can print out.

I used the setTimeout() function so that all my tweets would not fade into the screen all at once. setTimeOut() calls printTweet.

printTweet first pops an element from the end of the array. Then it creates a span element that will contain two inner span elements, one for the twitter user's picture and one for the tweet. Later in the function, I use .appendChild() to actually make the outer span tag contain the two inner span tags.

The tweets are to appear in random colours, depending on the mood typed in. The "sad" tag will appear in mostly blue colours, "angry" in red, and "happy" in all ranges. I specified the 10 colours that each of the tags will appear in and used the Math.random() function to get a random number between 1 and 10.

I then used the .css function to specify the appearance of the span tags that I created. (This is where the random font colour of the text is set, amongst other things.)

And then the fun part! I used .fadeIn to specify how long the tweets should take to appear on the page. And then I used .fadeTo to reduce the tweet's opacity!

Try it out! Happy coding!

Started @ CDOT

2010-09-10T15:18:00.001-04:00

This fall, I am a "Student Research Assistant" at CDOT. It's a wikkid title because it sounds like it belongs to a university student. However, as one of the administrators explained it, we are not researching for the sake of knowledge--we are researching for the sake of a final product.

The first thing I did was set up my workstation. Sounds easy enough, but when you throw in a MAC OS (which I'd never worked with before), it's new*2.

Some things I learned:

sudo su

1) I should've set up my account on this computer under the admin account in the first place.

2) I utilized the sudo su command while setting up my github account (as outlined in Anna's blog). (github is a version control system.) sudo is a command that lets you execute commands as a super user.

Since everything was done at the command line, as user donna, I was prompted for a password every now and then and then inevitably told that "donna is not a sudoer" (or something like that). The fix was to beat the cl to the punch and just type in sudo su and then type in the admin password, and off I went.

(Must mention that even while under Admin account, I had to sudo su in order to access other user's accounts).

github

As I already said, github is a version control system. In one of my classes, OOP344, I used Tortoise SVN, and that was fine because it had GUI. Since Anna's instructions were on the command line, that's how I worked, too. It was frustrating =\

After cloning the repos that I was going to be working with, I created a local and a remote branch (again, instructions are in Anna's blog), but I stumbled a bit. Things I learned:

1) When branching, make sure you are in the directory that contains the cloned repo. So when you clone popcorn (one of the repos I will be working with) like this:

git clone git://github.com/mozilla/popcorn-js.git popcornDonna

follow with cd popcornDonna, then create the branch (git branch firstBranch).

2) As Anna says in her blog, some errors that result in creating a remote branch (using the git push command) are fixable by editing the config file. She is referring to the .git/config file.

How to find the file: make sure you are in the folder that contains the repo you cloned (in my example, the folder is popcornDonna). Then type ls -a . One of the results should be .git. Change directory to .git (cd .git). Type ls and you should see the config file. Edit it.

So far so good, tho. Just going to be reading up on Popcorn and learning some JS for the time being.

Game development club at Seneca

2010-07-18T17:06:00.003-04:00

Did you know there's a game programming club at Seneca? Check out the unofficial website.

Actually, the club is in its neonatal stages. The founding students still have not found a faculty advisor and, from what I can gather, they are trying to get the Arts and Animation faculty in on it too. Sounds like a big project! However, if they can get enough students to join, the ball might actually start rolling and things will be organized in time for the fall semester (less than two months away!).

Hope to see some of you there!

Container Classes

2010-03-15T08:21:00.003-04:00

IRC meetings have become so slow that Fardad Soleimanloo has now decided to try using voice conferences through Skype! Though OOP344 is not an open source course (like OSD600), the project for it sure is starting to look and work like one. (Just this morning I had to chuckle at my team--scheduling a meeting that works with everyone's schedules is a nightmare.) I find the course incredible because in all my years of studying, I've never seen a teacher put so much time into a course or students asked to put so much into it either. In university for Economics, I mastered the art of cramming for every midterm and test (those were all you were getting marked on, after all). I studied strategically and aced most of them, but I didn't retain much info in the end. I'm finding that in college, with weekly due dates for each course, you have to keep at it or fall behind if you let things slide even by a week.

Anyway, the real point of this post was to share a couple of nifty little webpages about container classes. In our last IRC meeting, my group had a bouncy, yellow question mark over our collective heads because we just didn't grasp the idea of them soon enough. But the fact is, container classes really are how Fardad described: they are an array (another kind of container) for objects. Here is the webpage, complete with a sample class, from learncpp.com. I haven't read this one from parashift.com yet, but the writing seemed pretty funny, especially after learning the basics of a container class.

I haven't read the textbook section for stacks and queues yet (and yes, that means I haven't coded how a queue works yet, either), but the above webpages really helped me understand the basic idea for them. I hope they help someone else, too.

OOP344: Logic problem

2010-03-03T11:00:00.000-05:00

Part of my contribution to this first programming project was to code the section of the function that would actually let the user insert characters into a line. (Project specs here. The function I'm working on with everyone is bio_edit()). On and off during the week of the due date, I had successes, but the last hurdle stumped me. At some point in time, everything was working perfectly--it's simple logic--and then I just had to re-do someone else's code, and everything with regards to insertMode stopped working.

Someone suggested using memmove when shifting the string, but since I was (and still am) afraid to use a function I hadn't used before, I decided to stick to a for-loop. It didn't work as well as I thought. I coded it so that when inserting characters, everything from the end of the string to the current cursor position was shifted to the right, and the character typed would be inserted in the current cursor position (*curpos). However, the characters being shifted included those beyond *curpos, and any letters I typed were basically overwritten by whatever was to the left of the cursor.

I isolated this section still got the same results. My code looked like this:
Then I got it fixed with this:

Line 645 now takes into account *offset where it didn't use to (line 508 in the first screenshot). The really bad part about trying to debug this was programmer failure--I kept forgetting the fact that *curpos is related only to length of the editing field and not the str being edited. *offset + *curpos is the index of any character in str. Whenever I tried doing a walkthrough, everything made sense. At some point, I stared at my long enough to realize it. Massive failure.

Though the project has been submitted, I'm still going to attempt using memmove to reduce the number of lines. (That is, if playing Prototype doesn't get in the way, haha.)

OOP344: Creating an editor

2010-02-17T13:27:00.006-05:00

Before things get any more hectic with this OOP344 project (in which we're to create a programming editor, a la vi), a few things:

1) Compiling can be a pain in the butt

I have never compiled anywhere besides linux (on Seneca's matrix server) before, so learning to compile on Visual and Borland was a frustrating experience. More than anything else, it was being told that the library I was trying to #include did not exist.

To compile on Borland, I first had to create a batch file that would add to $path the path-file to my Borland compiler. I still have not figured out a way to set this permanently, so for now I must run it everytime I need to use Borland. It's a simple .bat file with one line:

set path=%path%;C:\Borland\BCC55\Bin

I had to create a bcc32.cfg file in the Bin. All it contains is:

-I"c:\Borland\Bcc55\include"
-L"c:\Borland\Bcc55\lib"

It cuts down the typing I have to do when compiling because it automatically includes the libraries I need to run my program.

Visual and Linux were easier, though with Linux, I forgot to "-lncurses" at first.

2) I'd been accidentally committing to trunk all along

What I'd done was copy and paste the PRJ file (which contains the .c's and .h's for the project) from trunk to my workspace. I thought that because it was in my workspace, committing any changes I made was ok. I was under the impression that anytime people committed to trunk, everyone's version of similarly named files in trunk would be updated, too. After all, how else is everyone going to get synched?

This issue has been fixed now. I've done a fresh checkout, and after all the editing I've done to my code, I'm going to be ready to merge my code with the trunk code soon.

Mystery solved

2010-01-22T21:56:00.003-05:00

Mystery at the HQ Solved!

Two of the three new Coding Rangers have been identified in class. From first impression, one looked a lot friendlier than the other, but yours truly is trying not to be judgmental because the wind blew the wrong way on her face when she was pouting. Could've happened the same to the other dude.

Coding Styles

The Coding Rangers had their first (informal) meeting last Thursday to decide on the most basic coding styles. We agreed on most topics without trouble, like tabbing instead of spacing and placing the opening curly bracket on the same line as the condition or function that opened it. (I like Cindy Laurin's style of placing the opening moustache on the next line by itself. I think it looks neater, but I was defeated in this discussion.)

My professors have tried to sell students on their preferences. Last semester, a classmate lost marks in an assignment for using spacing. Mark Fernandes told her to use tabs. Now Fardad Soleimanloo is telling us to use spacing. (I think his argument is that the length of tabs differ between editors.) Anthony Austin wants everyone to use vi and ssh. Use ftp or telnet and you're going to find "YOU R OWN3D" in your home directory. (He's pretty darn serious. He spent 10 minutes on the topic.)

I've personally made up my mind to use vi and do 2-space tabs for programs and spaces for html.

Mighty Morphin' Coding Rangers

2010-01-20T16:19:00.000-05:00

Trouble at the Coding Rangers HQ!!!

The Coding Rangers, formed on the first day of class and baptized two days later (thanks to Chris), announced that they are still looking for 2-3 members to join them.

And now 2-3 members have! The problem: yours truly have NO IDEA who these 2-3 members are. She suspects someone does, but no one is speaking up--YET.

The Coding Rangers are a group of OOP344 students who will be working together (OPEN SOURCE!) on a yet-to-be-determined project. They are an amalgamation of two pre-existing groups: Annie, Chris, and yours truly, and Dachuan and Shengwei. Strangers to each other still, they have yet to really communicate.

Stay tuned! See how this level 1 Communication Failure will be resolved (or not resolved--uh-oh!).