Scripting languages and KDE
Up to now, the main language in which KDE applications are developed is C++. This is a perfectly valid choice: however, at least for users that don’t have a computing background (I include myself in the list: I do science) C++ can be a rather steep requirement (although Hans, fellow staff member at the KDE Community forums, is doing a nice job in explaining his learning experience).
Scripted language bindings (that have a lower barrier of entry) for KDE exist: on top of my memory I can list Python, Ruby, and C# (and I’ve probably missed a few). Despite that, the only official non-C++ programs that are in KDE’s SVN currently, including only what’s in the official modules, are system-config-printer-kde and printer-applet, which are written in PyKDE. This is quite striking to me, because the language bindings allow full access to the API in kdelibs, and thus allow the creation of applications with (theoretically) less effort on part of the writer. Yet, such applications, despite the bindings being rather mature, are lacking.
Why is so? By applying Occam’s razor, Ithink that this is because most of the current KDE contributors are C++ coders. I seem to recall, though, a debate when system-config-printer-kde and printer-applet were introduced, where discussion arose between “pro-scripting” people and “anti-scripting” people. The major complaint is of course performance, as scripting languages are rarely at the level of C++. Still, I think that a more diverse language ecosystem would benefit KDE by making possible the arrival of new contributors that are not necessarily well-versed into C++. In some cases, such as Plasma, having programs directly written in scripting languages is a guarantee of less crashes dn more security. In the end, KDE would really benefit from it.
Before someone flames, let me state that I’m not advocating a reduction of C++ based programs in KDE. Just more choice. FOSS is all about it, right?
EDIT: After reading Richard Moore’s comment I thought I’d clarify my point. It is true that core modules need to stay C++ only to avoid more dependencies in the chain, but at the same time I feel that KDE does not stress enough that you can write applications (that can live well outside the core, for example in extragear) in languages that are not C++. This can be a limit for contributions from people that “do not speak” C++.
‘Why is so? By applying Occam’s razor, Ithink that this is because most of the current KDE contributors are C++ coders’
Before you can apply Occam’s razor, let’s recall it: ‘the principle recommends selection of the hypothesis that introduces the fewest assumptions and postulates the fewest entities while still sufficiently answering the question’. To apply this you need to consider the facts (all of them).
There are some very real problems with introducing additional languages into the core of KDE, many of which were discussed in the thread about the printer applet. The problems include the explosion of dependencies which impacts on packagers, and the ability to have a small but functional base install of KDE. There are maintenance problems as we no longer have a common language that we can all work on.
That said, most of the issues above do not apply if we’re talking about things outside the core modules. For example, Guarddog is written in python and worked nicely for many users. Choice is good, but we really don’t want to make users install half a dozen language environments (and their bindings) just to get a working KDE run-time.
@Richard Moore – If that’s the case, it’s fine if things should be outside the core modules, but the main issue I would like to raise is that (all IMO), KDE is not too welcoming for people using scripted languages, compared to what’s available for C++, which poses quite a large barrier of entry.
When I say that, I fully realize that this is also a manpower issue: the number of people working on bindings is not as optimal as it could be. Still, I think KDE should stress that to write applications C++ is not an essential requirement (unless you want to go into the core).
Well, I think some of it is the memory that the scripting languages take up that C++ applications don’t. For example, on my laptop, the printer applet is taking up more memory than plasma, kwin, or knotify4. On the desktop (the one where more apps are run and things get done), it is near the top at startup, but does fall down the list as the other apps I use use more memory (still 1% of the total RAM and ~3% of the RAM that is used). It is currently 11th out of the processes that are usually open.
Why must an application that just sits in the tasktray use up 15MB of RAM? When changing (or using for that matter) printers is rare? This, I think is a good reason for avoiding scripting languages for applications. Especially with lower capability computers as targets these days. I have 4GB of RAM to burn and I sit with 1.3GB used most of the time. What to tell users with netbooks or older hardware with 1GB of RAM?
@Ben Boeckel – in certain non IT environments having someone with C++ skills is quite rare (for example, my own field: bioinformatics), and those fields could be nice niches for KDE to fill using scripting languages. Why closing the door to them? About the RAM usage, I wonder when the impact is real and when is negligible – there is a need for some serious testing here.
@Einar: ‘the main issue I would like to raise is that (all IMO), KDE is not too welcoming for people using scripted languages, compared to what’s available for C++, which poses quite a large barrier of entry’
I agree potentially a real issue. Any suggestions on what we should be doing about it? We are currently working on plasmate that aims to make writing plasmoids in languages other that C++ easy. I think we’d be open to other similar ideas (or even better entirely different ones that we haven’t thought of!).
Which is precisely why people (myself included) do not write applications using the bindings. People can not be expected to have them installed by default, installing them can sometimes be a pain in the ass, packaged versions are frequently buggy, and even though Richard Dale and co. are doing heroic work, there is no assurance that the bindings will continue to be maintained well into the future (a small game I wrote with the Qt3 version of QtRuby several years ago is now basically unusable), and getting it to work on any OS besides Linux is a crapshoot. So even though developing an application using a higher level language and bindings may be faster and more pleasant, the drawbacks in terms of compatibility, support, and quality assurance would seem to outweigh this. (Or at least, I would expect more applications to have been written using them, if this were not so.)
Basically, the bindings are second-class citizens. If you want your application to be of consistent and lasting compatibility and quality, you are forced to write it in C++. I think it’s a huge shame. Mr. Dale and the other bindings developers are putting tremendous work into producing high-quality bindings, but due to these kinds of external factors, their work goes largely unused, if not unnoticed. It’s a waste.
What can be done about it, though?
Unless someone has a better idea, I think it really does come down to whether we are willing to “introduce additional languages into the core of KDE”. There is no purely technical solution to the above problems. Put the (high quality) bindings directly into kdelibs, the bindings for other modules directly with those modules, and also write an important application or two in kdebase using those bindings (similar to what Apple did with Carbon and the Finder*). This way, if something breaks with the bindings, it is no longer Someone Else’s Problem, rather we are essentially forced to fix it, and distros would be forced to package them well and ship them by default. As a result, developers could be assured that their application will run well on a wide range of users’ systems, that they won’t run into significant obstacles due to reasons outside of their control, and that this will continue to be so in the future. Basically, we have to make a choice: do we want people to actually use the bindings, or do we not? Because if we do, we have to make the sacrifices in terms of maintenance and dependencies mentioned in the above post. There is no free lunch.
* Yeah, the Finder is not exactly an attractive example, but I think the reason for that is that Carbon was an older, compatibility API (more like Qt3Support) — whereas bindings for higher level languages are forward-looking. And writing the Finder using Carbon did succeed in its intended purpose: making sure Carbon was well-maintained and assuring third-party developers that it was safe to use.
Alternately, we can say that the costs are too high, that we want to avoid distros having to package multiple bindings for multiple languages, and users running multiple language runtimes potentially leading to memory bloat. But in that case, we should make no pretensions that the bindings are first class citizens and just as viable an option as C++, and should not be surprised when people refuse to use them.
The reason why C++ is so dominant in KDE is due to project culture and its history. It has been a C++ project from day one and naturally attracted C++ experts and fans. This has projected the image that C++ was a requirement for doing KDE development even though high quality bindings have existed for quite a long time already. Referring to languages like Python and Ruby as being “scripting” languages also sends the wrong message and makes it sound like they are “development with training wheels on”. They are general purpose programming languages.
I find the dependency argument to be not particularly compelling for established languages like Python and Ruby. I don’t know any desktop distributions which don’t already package these languages.
But let’s not kid ourselves. We as a project are not in a position to choose what other people, often in their own space time, are going to develop their software in. For example, saying that C++ should be used, isn’t likely to convince someone to develop in C++. Instead they will most likely not develop at all, or choose a different platform. “Beggars can’t be choosers” as the saying goes. KDE should be open to examine and consider any software regardless of what it is written in. It isn’t a choice between an application written in $LANGUAGE or the same application in C++. It is simply “take it or leave it”. If you don’t want that $LANGUAGE application, then you go without it.
On a more positive note, I see a lot more development and small projects being started on KDE using languages other than C++ than in the past. The situation is improving, and we are seeing more people developing on the KDE platform who otherwise may have gone somewhere else. We can always use more contributors.
I’m also a bit concerned with the memory usage of having python, ruby and mono around for a complete desktop. I agree that C++ isn’t the easiest language, though Qt makes C++ a lot easier. People actually learned C++ by coding their first KDE app.
Developers also learn Objective-C just to code for the iPhone. :)
@Einar
> in certain non IT environments having someone with C++ skills is quite rare (for example, my own field: bioinformatics), and those fields could be nice niches for KDE to fill using scripting languages. Why closing the door to them?
I this is is exactly the target audience for scripting languages.
As a company you already have a lot of code in a certain language (e.g. python). With the KDE bindings this allows you to write a KDE frontend pretty fast, and hook into the existing code.
If you feel KDE is closing the door for this case, I think the devs love to know how.
@illissius
> there is no assurance that the bindings will continue to be maintained well into the future (a small game I wrote with the Qt3 version of QtRuby several years ago is now basically unusable),
Are you sure the bindings are at fault here? Qt 3 -> Qt 4 was a major upgrade of the C++ API’s as well. KDE 3 -> 4 in a large extent too. All C++ programs also needed to be ported. The new API’s of Qt 4 are expected to last a few years longer, and it has proven to be a very good base for the future.
I don’t see why the existence of Qt4 should mean that an application written in Qt3 cease working. Indeed, Amarok 1.4 still works just fine, and I use it every day. Not the case with my small QtRuby app. (And it’s really not that important, just an illustrative example.)
I think that this is because most of the current KDE contributors are C++ coders.
I suspect that most KDE coders are “fluent” in more than one language. C++ may be our lingua franca, but most of the developers I know ALSO code in Python, Perl, Ruby, or PHP. That said, my guess is that it’s not lack of developer support — it’s lack of documentation. The C++ API is very actively maintained and is fairly accessible (just go to techbase). I haven’t found very much documentation on the bindings. Now I’ll admit it’s been awhile since I last looked for API references, tutorials and so forth, but if I remember correctly, the only semi-decent documentation I could find was for the Python bindings and even there it wasn’t always clear how to mentally translate the C++ API into a form that was suitable in Python. I’m sure this area has improved a lot, but I bet it’s still a greater discouragement than lack of support from the KDE devs.
@Richard Moore: “I agree potentially a real issue. Any suggestions on what we should be doing about it?”
I’d say mostly documentation. For a few things I’ve been experimenting with, I had to read up the C++ sources to get a clear idea on how to proceed. More tutorials would lower the entry barrier. Also, perhaps, a comparison chart of what a language binding can or can’t do (KConfigXT comes to mind – you can subclass KConfigSkeleton but you can’t use .kcfg files in Python, for example) would be nice.
In this direction, I’ve recently ported one of the C++ program tutorials to Python and I hope to get it in Techbase sooner or later (the code is ported – I need to write the tutorial text).
@Simon Edwards – I used “scripting languages” as a – perhaps over- -simplification (I’ve been using Python for a number of years now and to me too it is a full fledged programming language).
Personally, (and this sounds like the general consensus) I think anything in non-C++ language bindings is fine outside of the KDE core. Basically I think there just needs to be some high profile projects written in the bindings to get people excited about writing KDE apps in that language. For instance look at what Banshee, F-Spot, and Tomboy have done for C# in the GNOME community.
@Harley – I would count Pardus Linux as a high profile example of what PyKDE can do, to name an example. But yes, single applications that show clearly how well bindings work (they do: the stuff I’m working on in my spare time works great thanks to KDE libraries, and it’s even lean, number of lines of code wise) would be definitely welcome.
Having been forced to use Perl, Python and Java on projects during the last few years, I’m a bigger fan of C++ than ever. For me, it’s not a question of being interpreted, compiled or bytecode-compiled, it’s more a question of being typesafe, consistent, powerful, readable and _constant_. There is a new kid on the block every few months, be it Vala, Groovy, Ruby, C#, Lua, Python, Perl, ECMA, PHP, or even VB. Many evolve fast. Some don’t have solid enough IDEs, some don’t have solid enough libraries, some are not solid themselves. It’s a moving target. I don’t think it’s a problem of KDE, imho it’s more of a problem of the languages themselves.
As a ruby developer who would love to write desktop apps in KDE, I’ve found that the biggest thing that prevents me from doing so are issues with interpreters support of threading. The ruby interpreter doesn’t support native OS threads (same applies to python I believe) which can cause issues with a framework like KDE which uses them a lot and also controls the main loop.
I’m more familiar with how these issues manifest themselves in GTK, where for example: segfaults can occur when trying to access GTK from the wrong thread, ruby code can be starved for cpu cycles, and long running ruby code can block screen redraws and button clicks. Last I checked KDE suffered the same sort of issues.
The next version of ruby (1.9) implements native threading I believe so maybe some of these issues will go away. I hope so because there’s been a lot of progress in automated testing using languages like ruby, and I would love to work on a desktop project with a full suite of automated unit and functional tests.
@robertm – I don’t want to sound like there’s lack of support from KDE developers. I just wanted to resume the discussion on bindings, which IMO are first class citizens but due to a series of (solvable) problems aren’t as used as they could be. The first would be bringing more people in, especially to write docs.
@Andy – Python has the Global Interpreter Lock which causes some problems with regards to threading (not yet removed and unsure if it ever will since there are a number of modules inside the core that aren’t thread-safe). I think it can be worked around, though.
An other reason is that despite all the bashing on the complexity of C++, the combination of Qt with a certain subset of C++ (which maybe we should call QtC++ :) ) make a very powerful and easy to use development framework, which reduce the need to go for an other language.
One also should note that they are very few new applications introduced in the main modules nowdays, because they are rather “complete” and already formed quite an advanced applications suite. So it would be worth to extend your search to extragear (and even to outside, especially with the move to DVCS people feel less interested in being in the official repository), there are also many PyQt4 applications, fewer RubyQt (despite being an avid C++ coder, I wrote two of them ;) but don’t really advertise them nor have taken the time to put them on a public repository…), and I have heard of a few Qt/C# ones.
@Cyrille Berger – My point, as stated before, is that people who don’t know / can’t learn C++ should be welcomed as much as the ones using C++. I heard myself of a number of PyQt applications, but not as many use PyKDE, for example.
Increasing the “awareness” for bindings will open up more fields for KDE: being biased towards the life sciences again, I’d love it make inroads in a field which is basically all Java and web-based applications. But in those fields, almost no one knows C++.
From reading the comments, there appear to be two separate arguments here. Writers who want the bindings included with KDE by default emphasize the need to make development easier. Writers who are against the idea point out the fact that Python, etc. use more memory and have more limited performance. Both arguments are essentially correct. But nobody is suggesting a re-write of Kate, porting it to Python. I think we can all agree that it would be a terrible idea to do so.
I do however think a LIMITED set of quality bindings would be incredibly useful. The two languages that come to mind are python and javascript. These languages are popular in the community and well supported by upstream development. Rather than focus on every possible permutation (Perl, Java, C#, Haskell, D, etc.) the KDE project should select 1-2 bindings to include in the kdelibs. This would expand the programming options within the desktop without causing a cascade affect dependencies. Of course there will be those who will criticize the decisions made regarding which languages are chosen but it would be a relatively sane middle ground to shoot for.
The Gnome approach, where C#, Python, etc. are constantly running for basic core functionality is ridiculous but there for smaller applications, python or javascript would be appropriate and would only affect the users of these smaller, niche applications.
illissius said: I don’t see why the existence of Qt4 should mean that an application written in Qt3 cease working.
Software doesn’t suddenly stop working for no reason. I think the problem you are talking about is caused by a change to a Ruby C function between Ruby 1.8.6 and 1.8.7, where the type of an argument changed from ‘char *’ to ‘const char *’ that meant that Qt3 QtRuby no longer compiled.
I do have a backup of my Qt3 based development environment on an external USB disk drive, but when I went to fix the bug, for some reason I couldn’t get it to work in a chroot’d environment. So I’m afraid I haven’t been able to fix the bug, but it is only about an hours work for someone familiar with C/C++. I really can’t see that as much of an argument for not using QtRuby. If you had written your game in C++, I wouldn’t be surprised if the code needed some similar small changes over the past 5 years due to changes in the GNU C++ compiler.
illissius said: there is no assurance that the bindings will continue to be maintained well into the future (a small game I wrote with the Qt3 version of QtRuby several years ago is now basically unusable), and getting it to work on any OS besides Linux is a crapshoot.
I’ve been working on QtRuby for over six years, and I think it is highly unlikely that I will give up after all that time. And there are other people who know the code, should I fall under a bus.
As for porting GPL’d Qt3 applications to Windows and Mac OS X, of course you can’t because there was no GPL license available. Porting a Qt3 QtRuby app to Qt4 is no harder than doing it for other languages, and may even be easier than in C++.