?

Log in

No account? Create an account
To The Side

The Problems of Perl: The Future of Bugzilla

Once upon a time, Bugzilla was an internal application at Netscape, written in TCL. When it was open-sourced in 1998, Terry (the original programmer), decided to re-write Bugzilla in Perl. My understanding is that he re-wrote it in Perl because a lot of system administrators know Perl, so that would make it easier to get contributors.

In 1998, there were few advanced, object-oriented web scripting languages. In fact, Perl was pretty much it. PHP was at version 3.0, python was at version 1.5, Java was just starting to become well-known, ruby was almost unheard of, and some people were still writing their CGI scripts in C or C++.

Perl has many great features, most of all the number of libraries available and the extreme flexibility of the language.

However, Perl would not be my first choice for writing or maintaining a large project, such as Bugzilla. The same flexibility that makes Perl so powerful makes it very difficult to enforce code quality standards or to implement modern object-oriented designs. Here are the problems:

Read more...Collapse )

Comments

Page 2 of 3
<<[1] [2] [3] >>

OO Exceptions in Perl


One can do Object Oriented Exceptions in Perl using Exception-Class or Error.pm. Error.pm provides a lot of syntactic sugar, but they're a huge leaky abstraction. E-C has no syntactic sugar and works better.




You can also mix them.

Re: OO Exceptions in Perl

I can take a look at those. They're not source filters, right?

-Max

(Anonymous)

Lisp

I have read some good things about Lisp as a development language, and am myself looking into it as a possible language for my next project. That being said, it has been almost twenty years since I last programmed in Common Lisp, and, while I was very impressed with it for rapid development back then, I am not sure how well it has aged (e.g., how well it supports sockets, HTTP, SSL, windows, etc.). Does anyone have any suggestions on the suitability of Lisp for modern programming projects?

Re: Lisp

Even if it is good, it doesn't really matter, because the language itself isn't popular enough to sustain an Open Source project like ours.

I also know of very few major applications written in Lisp.

-Max

Why not do those steps minus the language switch idiocy?

Dude,

Your problem is that you have a large and complex application, written by developers who weren't very familiar with the language they were using, and maintained by people who are even less familar with good Perl.

I imagine if you switch to Python, with little python knowledge you'll get to repeat the same newb mistakes in a new language while, losing the benefits of refactoring over re-writing.

Seriously, why on earth aren't you looking at modernising your Perl skills - If most of your Perl experience is with Bugzilla then that's got to be like a priest who's experience with christianity is Dan Brown books.

There are companies in the US and Europe that can train to a high standard in Perl, using the latest practices, and then hopefully you can start testing and documenting bugzilla and refactoring it.

Heck.. I was able to port a fair chunk of Bugzilla's core to Maypole in the space of a couple of evenings. You're already using TT (which by the way is unrivaled by anything in other languages), so moving to an MVC framework - or even something as lightweight as CGI::Application would save a huge ammount of grief.

Anyway.. the first thing you need to do is stop blaming Perl and start looking at why you let the code get in such a state.. it's not like Perl forced to write it badly.. Heck - it *could* force you to write it well using Perl::Critic, Pod::Coverage, etc.

Re: Why not do those steps minus the language switch idiocy?

I like the Dan Brown quip, but it's more like a priest whose experience with Christianity comes from reading the Bible and adhering to its principles. Then the priest complains about how wacky it is, and an apostle says "You learned from the *Bible*? There are a shitload of new epistles, aren't you on the mailing list? Don't you read ChristianMonks.org daily?"

(Anonymous)

Your competitors aren't using Perl? Really?

almost all of our competitors have one advantage

Hmmm...RT (http://www.bestpractical.com/rt) is written in Perl, and doesn't seem to have the problems you discuss. Also, rather than using mod_perl, think a bit outside the box, and look at FastCGI as an alternative like your competitors and other frameworks have.

Re: Your competitors aren&#39;t using Perl? Really?

Yes, RT is also not ahead of Bugzilla in terms of features in any sense. :-) It also has existed just as long as we have, or longer.

I personally really dislike RT, but that's a personal opinion.

-Max

(Anonymous)

mod_perl tuning

If you'd join us on the mod_perl mailing list, I'm sure we could cut down the memory footprint. Any persistent web environment will have a relatively large footprint, but a lot of tuning is possible. There's plenty of advice on this available for free at http://perl.apache.org/.

Re: mod_perl tuning

Actually, I did ask on the mod_perl mailing list, and the answers were mostly, "That looks normal to me" or "Yep, nothing can be done about that, really."

-Max

(Anonymous)

mod_perl perl and frameworks

You can tap into Apache and create a handler, then cache standard data within apache itself. The others that mention mod_perl tuning are correct. You should read a bit more. Have a look at Catalyst, or even Slashcode itself. Big install, sure, but how they've both done things is slick.

Re: mod_perl perl and frameworks

We do cache standard data in Apache itself, when using mod_perl.

I've been looking into Catalyst. I might look at Slashcode also.

-Max

Python

"Not having curly-braces on "if" statements and other blocks makes it hard to figure out where you are in the block structure without a special editor to help (like Komodo)."

If you use tabs or few spaces then you will easily see the block structure of the code in any text editor ;)
#######
"Poor Unicode handling"

To be changed in Python3000
#######
"No standard way of installing modules like CPAN"

well, Cheeseshop/eggs is a "standard", and a lot of linux distros pack python modules on their own (if they are used for something) So if you make a popular software in Python then by adding it to Linux distros all dependencies will go as well.

##########################
Pylons or high level Django are very good frameworks and are able to spawn a good bug tracker.

Re: Python

Yes, but we don't use tabs and tabs are generally considered bad programming form because they don't survive many different text transitions such as email.

Even with four spaces, the block structure can get really complicated--you're in a class, then you're in a function, then you're in for loop, then you're in an if statement, then you have some more stuff after the if but inside the for...it just gets complicated to read.

I hear great things about Python3000, but it's too far away for our needs, as I understand it.

It's true about the distro packages for Python, that really does help.

And yeah, I've looked at both Pylons and Django. It seems like Pylons is a bit more "don't get in your way" and Django is a bit more "build a website, not a shippable application," so I'm suspecting Pylons would be overall better for us.

-Max

Don't burn your bridges.

I would strongly suggest that a lot of what you have to say about perl is due to an outdated view of the language. You should be able to incrementally modernise the bugzilla code base with the help of Catalyst (http://catalystframework.org), new OO libraries (Moose - perl 6 objects in perl 5) and a decent ORM (your own, or DBIx::Class). An excellent testing/deployment platform - i.e. catalyst's built in server and the option to serve with mod_perl, or better, FastCGI. Also you can distribute your catalyst application as a binary executable (make catalyst_par) so you also win on reducing deployment headaches.

Re: Don&#39;t burn your bridges.

That's cool to know about the binary executable for the Catalyst app--that might be a big plus!

I've looked at DBIx::Class, and it's all right, definitely.

-Max

If my opinion still matters...

Yes, you got my motivation for the original port from TCL to Perl right. I knew that nobody would hack on TCL, and that tons more would use Perl.

These days, I would pick Python and Django.

However, I don't think you can really blame Perl itself for bad code. It is possible to write FORTRAN in any language. Some languages make it easier to write bad code than others, but it's ultimately always up to the programmers themselves to write good code.

- Terry

Re: If my opinion still matters...

Yeah, I completely agree. :-) In the balance of things, it's much more the individual programmers than the languange.

It does help if the language, the libraries, and the general community encourage readability, though. It's true there's a subset of the Perl community that works very hard on excellently-written, well-architected, very-readable code, but it requires an abnormal amount of Perl experience to be able to do that. :-)

And I agree, I'd pick Python if I were writing Bugzilla today, although I'd lean more towards Pylons since it's a bit more flexible for a tool like Bugzilla.

The advantages of something like Django or Pylons may themselves be enough of a reason for a port. But we'll have to see.

-Max

Perl2Anything

OK, I have to come clean. I absolutely hate Bugzilla. It ranks third in my list of annoying defect management tools behind IBM mainframe stuff (easily #1, particularly when you're not working on the mainframe) and Rational Rose. Well, I guess that makes it number 2, then.

However. Lots of people use Bugzilla. Much as I would like to persuade them to use almost anything else, I appreciate your aim to make it a stable platform for the future.

Now, I haven't really thought about this very much, but have you considered SWIG? I know it's typically used to offer statically-typed languages such as C++ or Java a binding into dynamically-typed languages -- and Python seems to be a current favourite -- but I see no reason why it shouldn't be used to offer a binding between, say, Perl and Python/Ruby. I believe it would be a useful bridge to wherever you want to go. You might need to add a few C/C++/Java/whatever wrappers, but at least this gives you the opportunity to, ahem, "refactor." Keep what you need and drop the rest, incrementally.

Just a thought. Probably a stupid one.

IMHO, 80K lines of code is hardly unmanageable. There's gotta be some way of chunking this stuff up.

Re: Perl2Anything

I hadn't really thought of using something like SWIG--usually I see C projects using it. It's definitely something to consider!

I agree, 80K lines isn't unmanageable, but it's kind of hard to manage as just me and a few other part-time developers. :-)

-Max

(Anonymous)

Great work!

Hi Max,

Thanks for writing a thoughtful article on the possibility of moving Bugzilla to another language. Your experience with Perl gives your assessments more weight.

When contemplating a language switch, also consider the likely future popularity of the candidate languages. The more popular a language, the more cool tools and libraries it will have. Popular languages offer a bigger pool of developers to contribute to Bugzilla. The use of Bugzilla would increase too, because developers are more likely to use a tool, such as Bugzilla, when it's written in a language they know.

It looks like Ruby is the scripting language of the future. In terms of industry dominance, Ruby is the next Java. Perl is in decline. Python, while a good solid language, has a quirky block and OO syntax that bother developers who prefer C-like languages. In any case, Python's growth is slowing and will likely peak in the next two years as more people discover Ruby. Although Ruby is not yet mature, the tremendous amount of effort now going into it now will bring it to parity with Perl and Python in the next 12-18 months.

D is also an interesting possibility. The vast majority of programmers prefer C-like languages. D is a C-like language that adds OO and garbage collection to C, but without the mind-numbing complexity of C++. C/C++/Java/C# programmers can easily become productive in D. It's possible that if Bugzilla chose to rewrite their app in D, that choice could put D on a path to become the world's preeminent compiled language. That's because Bugzilla is one of the world's preeminent open-source applications.

I think Bugzilla would be best served by being rewritten in Ruby or D. Python wouldn't be a bad choice, but it wouldn't be as good in the long-term as D or Ruby.

Re: Great work!

I agree with you about the future of Ruby. I'm not sure that Python will peak and die out, though.

I don't see any future in D yet. It doesn't have any momentum with non-CS-types.

Thanks for the comment, I really do appreciate it. :-)

-Max

(Anonymous)

We done did this (40,000 lines of perl -> ruby).

The Metasploit Project spent the last two years turning 40,000 lines of Perl code into Ruby. Ruby is easy to learn for Perl developers, is powerful as hell, is easy to read, and its not Python. Perl is a dead duck at this point - not many new programmers are learning it and all of the Web 2.0 shiny is happening with Ruby, Python, and .NET these days. We accomplished this with a parallel rewrite and a 6-month alpha/beta period. Once the beta period started, we stopped maintaining the old Perl version.

The new version can be found at:
http://framework.metasploit.com/

Re: We done did this (40,000 lines of perl -&gt; ruby).

Wow, cool! I was wondering if anybody did it.

What advantages did you see in general? And how did you convince people to do the rewrite? :-) Also, how many developers are on the whole team?

-Max

(Anonymous)

Hmm what about C?

I am not very good programmer, but you sound to me like - "I will do my best to not end with C++/C#"? So what about pure, simple C? With one plus = using Visual Studio for Development platform? You will have a lot of pros and only one cons - MS :):)But you can use MS V Studio like good tool with security templates, UML integration, etc. and by this eliminating (:):):)) this con!

Re: Hmm what about C?

Well, no, C would make all our problems ten times worse, and using Visual Studio would definitely be a bad idea--most of our developers run Linux.

-Max

(Anonymous)

Other bug trackers? Worth _having_?

> Nowadays, almost all of our competitors have one advantage: they
> are not written in perl. They can actually develop features more
> quickly than we can, not because of the number of contributors they
> have, but because the language they're using allows it. There are
> at least two bug-trackers that I can think of off the top of my
> head that didn't even exist in 1998 and were developed rapidly up
> to a point where they could compete with Bugzilla.

Care to name them?

I have seen a lot of bug trackers, and I would generally divide them into two categories. The first category consists of Bugzilla and forks thereof (e.g., Issuezilla), and the second category consists of lame half-baked immitations that are painful to work with, poorly designed, feature-impoverished, and terrible in terms of usability.

Particularly bad are the ones written in C++, PHP, and ASPX. Those languages should certainly not be considered.

The ones written in Python tend to suffer from the same malady that afflicts Mailman, to wit, if there's One Right Way for the programmer to do something in the language (Guido's Way, presumably), then the programmers tend to adopt the position that there's One Right Way for the user to want to use the software, the result being software that is painful to have to use if you don't happen to think exactly like the developers. I suppose it would be possible for the developers to overcome this and write good bug tracking software in Python, but I haven't seen an example yet. (Note that I'm not saying no such example exists, only that I have not seen it. Feel free to point one out, if you know of one that I've missed.)

Ruby would be interesting. Five years ago I'd have said it'd be impossible to find enough developers who know it, but Ruby has been receiving a lot of positive press lately (including in the Perl community -- a lot of the big names in Perl say nice things about Ruby), so its popularity has really grown. These days writing something like Bugzilla in Ruby might be quite feasible and worthwhile. I'd be interested in seeing that.

I wouldn't want to see development of the existing Bugzilla codebase cease, though.

And yes, Perl6 would be a great choice if a fully usable version of it existed today, but admittedly that's a pretty significant caveat. When choosing between Perl5 and Perl6, I keep finding myself picking the one that's available and widely deployed and stable, despite the philosophical allure of many of the design improvements in the new version.

Re: Other bug trackers? Worth _having_?

Hey, thanks for the very insightful contribution, there. :-) And I'm really glad that you feel so positively about Bugzilla vs. its competitors, that makes me happy. :-)

I think most people tend to prefer Bugzilla, also, but they sometimes install Trac because it's easier to install, not necessarily because they like it better. (Also because it does two other things--track SVN and it's a Wiki, so people like that. Bugzilla on the other hand will always stay focused on just being a bug tracker.)

Since we're way down here in the comments, I suppose I'll just say that the other bug-tracker I was thinking of was Jira. (Which, by the way, if you think about it a bit, is the end of "Gojira," the Japanese pronunciation of "Godzilla," and...yeah.) I haven't used it extensively, but I know there are some projects that are using it and like it, and it definitely didn't exist in 1998.

And yeah, I don't think we'll stop development of the existing codebase. There might be enough modules in CPAN available to make our development easier without having to switch languages.

Oh, and as far as Python having a One True Way, I do understand what you mean. :-) That's one of the things I like about Pylons, though--it's designed to allow you to pick the best tool for the task, so you aren't locked into just One Way.

Oh and yeah, I can't even imagine writing a bug tracker in C++ or ASPX. I could imagine writing one in PHP5, but only with extremely careful code review from highly-trained PHP developers.

-Max
LOL.

-Max
Page 2 of 3
<<[1] [2] [3] >>