Finding C Style typecasts During C++ Compiling

(As of 2013/05/11 there is a quick addendum at the end of the post)

This is a very quick post because it’s a very simple update to two posts from quite some time ago. I had discussed how to find C style casts in C++ source code and at the time of the first post I was thinking in casting to pointers. Then I made another post with to note that fact and a possible way to find all.

Well, there is a much better way. What is it ? With the g++ compiler you let it show you, that’s the way. How do you do it ? Use the option to g++:

-Wold-style-cast

and recompile all object files (if you use make you may very well have a make target called ‘clean’ then you could just run ‘make clean’ and then ‘make’).

g++ will then show you where every old style cast is used in the project’s source tree (that it encounters). Now it’s simply a matter of determining which new cast type you should be using for the specific cast, fix and recompile again. It’s that simple.

Addendum: Okay, to be truthful it is not entirely ‘that simple’. Yes, this is how to find the old style casts in YOUR source code but it should be noted that there are some system calls that are implemented as #define macros and do use the old style casts. There’s nothing you can (or should do) about those. Examples that come to mind are:

FD_* macros used for the select(2) system call.

WEXITSTATUS (and most likely the other related) macros for the wait(2) system call.

That all said the option is useful to use at times if you want to be sure your own source code does not use old style type casts.

 

Rest in Peace Jeff Hanneman

This is obviously not a technical piece but it is something that is important for me so I’m writing it before I forget about it. I regret that I missed several other significant deaths including indeed Ronnie James Dio of Black Sabbath (and others) and more recently Jon Lord of Deep Purple. Today is yet another sad day for metal music, an important part of my life.

I am even more upset now that health prevented me from seeing Slayer when I actually had tickets (in 2011) because Jeff is now gone. He was an amazing guitarist and his music will be well missed. The fact he died from liver failure and that I have a very close friend with liver disease as well as having lost family to liver disease, this really hits me hard. I typically do not think of the past and although that may be sad at times (hard to think of good memories in the past without thinking of the past) I would say it generally helps me cope with losses.

There really is not much else to say as this is truly a horrible loss to the metal heads of the world.

R.I.P. Jeff and thanks for your time and dedication with Slayer.

Solution: warning: do not list domain example.com in BOTH mydestination and virtual_mailbox_domains

It has been a LONG time since I wrote anything here and I have updated very little elsewhere (I did update some of the documents at http://docs.xexyl.net and I changed the main menu here a little bit but that is about it). The reason for such is a lot of health problems and such is life. While there’s a lot of stuff I have thought to write about the truth of the matter is they are quite low on my priority list. Nevertheless, since the topic in question is something easy enough (read: it’s not a lot to cover) to write about and since I have seen many people complain about the very warning by the Postfix mail server yet not find a solution I decided to spend a few minutes to write how to solve it. The complaints that are heard are typically along the lines of two themes :

But I do not have my domain in both mydestination and virtual_mailbox_domains!

and

There’s no explanation on how to get rid of this warning (aside from the ‘work around’ option to disable warnings) in the documentation.

Well, each has its problems but I will sort this out for those who have the warning I refer to. For the first problem, you must understand that just because you do not specify mydestination does not mean it is not set. Further, there is the chance there’s more than one instance of Postfix running (though to be fair this is less likely the problem). The second problem is not entirely true although maybe it should be made very clear in the documentation. That’s not up to me though.

So how do you solve the problem then? First, we’ll assume that your domain names are two of the for-documentation purposes domains:

  • example.com
  • example.org

We’ll also assume that the mail server is specifically example.com and you want either both to be a virtual domain or you want example.org to be the virtual domain. Choose your poison when reading this as it really does not matter in the slightest with this configuration.

So given the above parameters this is what you need to do :

  1. The server’s name should be what mydestination is set to, so in the configuration file (main.cf which is generally under /etc/postfix/main.cf) you need the line:
    • mydestination = example.com
  2. Next, assuming you have you your virtual domains file (a plain text file, one entry domain name per line) named (also in /etc/postfix/) virtual_domains then you should (adjusting to whatever your file name is) have the following in your main.cf file :
    • virtual_mailbox_domains = virtual_domains

    Note that if your file is actually a database file then you would need to adjust the line above to let Postfix know this (e.g., prepending ‘hash:’ without the quotes to virtual_domains and if you have updated the file do not forget to update the database file with for example postmap). It should go without saying but example.com (the final destination) should NOT be in the virtual domains file.

  3. The last step is essentially masquerading mydestination as a virtual domain (in the sense of the effect of the end result). Here Postfix is informed that local transport is to use the virtual transport service and that the local recipients are included in the virtual domains. That is possibly not the best way to word what is going on but the two lines should show all that is needed to understand it :
    • local_transport = virtual
    • local_recipient_maps = $alias_maps $virtual_mailbox_maps

    Assuming that Postfix virtual mailbox and virtual domains already worked then this, combined with reloading or restarting Postfix should be all you need to do. There is a way to ‘force’ Postfix to issue the warning (in the case that it’s not configured properly) but I do not have the motivation to write about it.

Hopefully this has been of help to some people and to those who I have left some questions remaining, I am sorry in advance. The real key is that your server name is the final destination and it is masqueraded as a virtual domain (but actually it is not in the truest sense of the word). This does however mean you might need to set up local users mail if they are to receive mail. I’m afraid you’re on your own there but if you only wanted virtual domains this should not be a real problem any way. Lastly, I am sorry if I have left anything out. As I said I have been quite unwell for some time and I may very well have neglected to mention something important. No idea when I’ll be writing next. Until then, so long and thanks for all the fish (yes, a little tribute to Douglas Adams as I missed his 61st birthday in March).

The Xexyl CentOS RPM/YUM Repository

Update on 2012/10/15:
I fixed an unfinished thought regarding installing CentOS (below). Indeed I have not updated anything of late as I have been quite unwell. There has been a more recent update of GCC (if I recall) but I’ve not been well enough to do anything with it for the Xexyl repository. I did at one point make a static package for gcc47 (so you don’t have to use shared libraries). Also, although the backport is potentially useful, there’s another option too. Depending on the environment it may actually be better. If you do have a need for more recent GCC releases and you prefer using just ‘gcc’, ‘g++’, etc., or even just want the system as ‘clean’ as possible, you could use VirtualBox and install the latest Fedora (which will have the latest GCC, generally speaking). It also allows for a completely different environment in the server, so that everything is separate. Otherwise, you are welcome to use my backport (that’s what it is there for). Lastly, I do not know when I’ll be writing again (due to health) but hopefully sooner than later. There have been several things I’ve wanted to write about in the past 3 months but I’ve just not been up to it.

Update on 2012/07/11:
I now have built the packages for i386/i686 architectures. So that means whether you use CentOS 6.x x86 or CentOS 6.x x86_64 you can make use of the packages. I did notice yesterday that CentOS 6.3 is out so I will likely build for that too in time, but note however the packages work for 6.3. as is so it’s not high on my priority for the moment.

I will document this and put it in a specific location and update this post when I do, but I wanted to introduce everyone to the Xexyl CentOS 6 RPm Repository. Yes, it’s true – you can now install gcc 4.7.0 under CentOS 6 in a safe way (I stopped working on CentOS 5 because I don’t have a need for that now. Considering CentOS 6, 6.1, and 6.2 are already out, I would highly suggest anyone installing a new CentOS install to use the latest). Not only that, you can do it without having to build the RPMs yourself. Indeed, thanks to a good long time friend of mine, I have a CentOS 6 RPM repo (along with yum files so you can easily update) and it is hosted on 2 servers each with a much faster connection than mine.

There is a cron job on the servers to sync from my server. It’s a nightly cron, so if I find any mistakes, or more likely – an updated GCC – it will be quickly adjusted (as long as I’m well and able to build it).

So, quick run down of how to get the Xexyl Repository installed :

# rpm -Uvh http://rpm.xexyl.net/redhat/6/noarch/RPMS/xexyl-release-6-1.noarch.rpm

That’s pretty much it. You should be asked (when first installing a Xexyl RPM package) if you want to install the GPG key and that is generated specifically for this reason. It means you can make sure the software in the repository is signed with this key (and the update program will make sure, as long as you don’t disable checking of the GPG key). So in short, just accept, and then you can install the programs via yum. Alternatively, you can install specific rpm files from the repository via yum or rpm.

Now, one last thing for this. How do you install gcc 4.7.0 in CentOS 6.x? After installing the Xexyl Repo package, simply type this command :

# yum install gcc47 gcc47-c++ libstdc++47 libstdc++47-devel libgcc47

That should be more than enough to pull in everything you might need. Oh, and yes, I should say that this allows for dynamic linking; indeed, you don’t have to statically link your binaries with this thanks to the GCC ABI policy: it allows for newer versions to work for older versions. Note you don’t need this if you’re in Fedora because Fedora already has the more recent gcc packages and indeed it will conflict with such.

Millennium Technology Prize

Hopefully my server can be reached globally. That is a different story though, and one I’d rather not get into (thanks, ISP!). Indeed my IP has changed and it’s not even static now. Anyway, I have some better things to write about than that.

So, back in April I read about Linus Torvalds being nominated for the Millennium Technology Prize. I thought that was cool and he had a good chance of winning. Well, yesterday, it was announced that he DID in fact win. Not only did he win, he shares it with a scientist who studies something else I feel passionate about: stem cell research (generally I’m for medical research – we’d not be here if it were not for it and the fact stem cell is only 10-15 years old means it is still a very early thing. Sadly many think that’s enough time but they just can’t or don’t want to understand and realize how long other illnesses have been around and the fact we’re still not even close to “curing” them).

So, given that I’m a huge Linux fan, and have used it for close to a decade and before that used SunOS/Solaris, FreeBSD and other Unixes, I figure I would reflect on some things. I used to ‘hate’ Linux; I loved Unix but not Linux. That may seem a bit odd but let’s just say I fully admit I’m a bit closed (okay, okay! Very closed) minded about a lot of things. There is however a funny story about how I opened up to both Linux and also OOP – object oriented programming. Back somewhere in the early 2000s I had a project that mainly developed with Linux. Now, I was a developer but I had FreeBSD as my primary programming machine. I even contributed a patch to the linux_logo program to make it FreeBSD compatible. Still, I didn’t like Linux that much. But there’s something called priorities. So, considering that the project I was part of (and to this day still am) was using Linux, I actually was too. I just didn’t like it as much (in short : it was different and being rather closed off, it must not be that “great”).

Firstly though, there’s two computer languages that I’ve been the most fond of, of the 10+ (likely more, but I’m not going to bother counting and classifying which ones I used to which degrees) I’ve used. As one can imagine, one of them is C. The other is assembly. So, in other words, lower level (and low level) languages, both having direct access to memory etc. And it happens that this project I referred to was in C. It, I might add is a multiuser dungeon (MUD) – a type of game, essentially a MMORPG but all text [remember text games? I sure do] – that originally came online in 1994. Yes, its still around and yes I’m still a developer and in fact am rewriting it in (the horror…) C++ from scratch (yes, that’s the project I’ve referred to at times in the past year or so).

With that background and terrible attitude, how did I manage to prefer Linux and also like object oriented languages, perhaps more than the structural/procedural languaes ? Well, at some point, a good friend of mine – Martijn – one of the founders of said MUD above – wrote some psuedocode but it was a cross between C and C++. Of course, being the type to say something, I did. I don’t remember the exact words but it was something like how is that valid ? (In fact: in C it is but it is perfectly valid in C++). But I was procedural loon! I then made the even bigger (some might call this a stupid mistake; I say it allowed a huge revelation to start revealing itself, and a good one at that) wise mark. I said something like : Even Linus Torvalds doesn’t like C++ as its too bloated. Yes, that was a VERY ignorant remark on my part. Oh, sure he said something like that in the past, and it has good intentions: C is not a bad choice for, say, an operating system kernel. Still, that does not mean C++ is ‘bloated’. Even if it is, that doesn’t have to be bad bloat and some might see it as feature rich (then again, some seem to think the C++ standard library is lacking and is too slow in progressing. I beg to differ but I also realize how developing something that needs to remain stable and clean will work and progress and C++ has some pretty nice features that make things lacking easier to do). So, back to the story. What did my friend have to say about my ridiculous remark ? He said (know that he wasn’t insulting Linus and most certainly he loves Linux – perhaps more than me) something that rather put me in my place but in a good way. It was so good it’s when I started to realize how closed off and how much I was missing. He was blunt yet accurate. It was a good way to address it, especially for someone like me – a quite sarcastic and satirical person with a big mouth. He said the following:

You mean the same guy who created the bloated Linux kernel ?

Well, of course, I wasn’t stupid. He had a very good point. Yes, it is true that the bloat he refers to is support for different types of hardware and a lot of customizable options and many other features that are actually good features. Still, it is a VALID point too. It opened my eyes. Then, over the next 3-5 years I started loving Linux, far more than I ever did the other Unixes, and then I also started opening up to Java, C++ etc. That’s where I am today. While C and assembly has a lot to offer, and while it is dear to my heart, that doesn’t mean other things can’t be good or dear to my heart. That’s something that everyone should try to remember. Besides that, as they say, ‘good comes with bad’ (and vice versa). So, shortly, some times things aren’t exactly as you might seem to think. It might be better, it might be worse. It might not. But it does a person well to remember that we’re all different, we all have our quirks and we all have unique traits. I’m not only thankful that I’m willing to accept when I’m wrong (and willing to learn more as a consequence, no matter what I knew or didn’t know before), but I’m also thankful that I had a big mouth and actually said such an ignorant thing. Equally, I’m thankful for my friend putting me in my place: I’ve learned a lot from him, gained wisdom and also been given the opportunity to experience much, much more (even working on his MUD is a huge thing but his friendship and life experience is too).

Yes, this post is intended to be about Linus Torvalds and his kernel. But I think that’s a very relevant story and its a way to show that what may seem ‘evil’ or ‘bad’ may not be that evil or bad after all. Linux does bring a lot of people over the world together, a lot of awesome groups, team work, and many other things, not to mention the possibility for some very very powerful and flexibile operating systems. It being based on Unix makes it even better. As I often say, I can do so many things with a ‘simple’ command line under Unix and especially Linux (GNU extensions rule!) that you either have to do complex scripting for or get an entire program (which may not be free) in some other environment. That is awesome.

So, I extend my congratulations to Linus completely. You deserve it 200%. Thank you from the bottom of my heart, Linus, for the amazing work you do, be it the Linux kernel, git, whatever. You have no idea how much you help people (actually you probably do but the point is I cannot thank you enough).

Cheers and congratulations to Linus!

The Impact of Fear, Uncertainty, Doubt

Note that this is some what of a quick write up (as in, it could be better organized). I have a lot going on, but as you’ll see, this has been delayed longer – much longer – than I originally anticipated. I think (hope) the ideas come across fine, and its of use or interest to someone. But if not, it might provide a bit of history about computer security (specifically related to malware).

Originally this was intended to be about the spreading of fear and misinformation about ‘cyber war’. And although some notes may be legit, a lot are unfounded and nothing new. Just because an agency never knew about something, does not mean it something majorly impressive or new. Sure, it could be. But that doesn’t have to be the case, and sadly, often it is not the case. Of course, many are drawn to fear, the uknown, and this makes them easy targets for propaganda and the like. I’m not going to cite examples, but they are out there. And although the concept of FUD was originally used in computer hardware sales, it can have a variety of uses, and computer security is one of them. This isn’t all about FUD but it does bring the spreading of fear, uncertainty and doubt, for gains or otherwise, whether they realize it or not.

However, I didn’t really write about that, and saw something else. Yes, it’s about technology and indeed security. But this time specifically about malware. It – according to some – seems like there are ingenious ideas coming out all the time, with respect to defeating some system. The fact is, while it does happen, it isn’t nearly as wide spread as some would make it seem. For example, I read a lot recently about how (e.g., Stuxnet and now Flame [the latter is what this is about]) can spread via a USB drive. Okay, so while this may seem so brilliant, it actually isn’t. Yes, it should be noted, but it’s nothing special either. I think it would suit these people well to either look up, or try to remember a bit further back in history. For instance, a USB drive can in a lot of systems be used as a boot drive. Now, that implies it has a boot loading mechanism. Well, what about the very old (we’re talking mid 1980s! Old for computers) Stoned and Brain viruses? Do you know what they attached to? Yes, boot sectors. Imagine that. Nothing too ingenious about a USB drive infection then, is there? And what about if it was attaching to a file on the drive (as after all, a drive has a file system, if its going to be used to store files)? Well, let’s see. That’s nothing new. File infectors have been around for decades. And then there’s the idea of file and boot sector (along with the master boot record) at the same time. That is known as a multipartite virus. Then, if it spreads via network, that’s nothing new either. Remember the Morris Worm? How do you think that spread? Exactly, via networks. Of course, there’s also combinations of all the above examples, and more.

Now, the malware Flame was recently uncovered. Some make it out to be a huge thing. And while it may have a lot of features in one program, that doesn’t mean its special, especially not compared to other major developments in the malware scene. Every time something new comes out, does not mean its impressive or a significant change. I can name many many bugs that seemed ingenious at the beginning, and [they] turned out to be nothing THAT original or significant. The problem is, people tend to be fearful, and also do not learn from history. That includes governments, and unfortunately that also is related to security (or lack there of). All these things we hear about, most of the time it isn’t new but old. I’ll elaborate.

In the late 1980s, a certain virus was found ITW (in the wild). Unfortunately for victims of such virus, it was very quiet, all the while slowly causing damage. You had backups, right? Well, in this case, it might even prove a problem if you do backup regularly: the fact its payload was done slowly and quietly means it would not be realized until backups have only the damaged copies. For instance, if you start out with a full backup, and have nightly incremental backups, and every fortnight you do another full backup, and then you rotate out older copies as time goes on, what might be the case when the damage is now visible? Essentially, the backups might only have the corrupted versions. And although I never liked (and still do not like) destructive code, I must give props to Dark Avenger’s virus (known by the same name as his handle), as that was quite clever. And if that wasn’t bad enough, there was another interesting feature, so interesting that it is actually a known concept in the antivirus and provirus scenes. The concept is piggybacking. What is that? Well, here’s the idea: First, know that this virus infected files but it added a twist: it infected files as they were opened. How, if the the virus is not running?

First, a bit of background about a programming concept. TSR stands for ‘terminate but/and stay resident’. What that means, is that it traps interrupt (or interrupts), which is basically an event. At the low level, that is to say, close to the hardware (e.g., your CPU), when a program requests to write something to the disk, an interrupt is called. The same applies for opening a file. The same applies to a lot of things in the computer. So then, how did Dark Avenger manipulate this? It went TSR (terminate and stay resident, i.e., it stays in memory after doing its work which includes installing interrupt handlers to certain interrupts). So, what did it hook? Interrupt 21h which was used when a file is opened (and other actions, too). To this end, it would piggy back on the antivirus. Indeed, this means as the antivirus is scanning the files for viruses, if it didn’t have a clue about Dark Avenger, and it was resident, then every single file that could be infected, that the antivirus opened to scan, would now be infected.

Here’s one of my favourite examples, and this is indeed (whether intentional or not) spreading FUD. Anyone remember that very destructive, wide spread virus called Michelangelo from the early 1990s? It seemed that everyone who knew about computers back then, was terrified of this oh so dangerous virus. Well, of course, since computer malware wasn’t new by any means, antiviruses made use of this scare (and admittedly, others). The media had a lot of articles about it, too. Sales soared. But not much really happened and it wasn’t really wide spread was it? No. And guess what happened after that? Yes, of course, some companies went out of business. That’s one example of abusing fearful people, if I do say so myself.

Back to Flame though. I’d just like to say that although it has a rich set of features, it isn’t really that ingenious. Think about it. They say it has a keylogger, sniffs the network, captures other things, and spreads through various ways (as I touched upon earlier, nothing new). Okay, some might argue: “It’s all in one program though!” True, but the only real issue is it is automated. There exists pentration testing operating systems, full of feature-rich tools, from port scanners, sniffers, and all sorts of other goodies. The interesting thing is, Flame is about 20MB. That is rather large. Okay, it isn’t large by todays disks, but for what it does (and is known), it is still fairly large. It certainly wouldn’t fit on a boot sector.

The interesting thing is, what Alan Woodward (yes, again) wrote about Flame. I’ll quote and remark.

This is an extremely advanced attack. It is more like a toolkit for compiling different code based weapons than a single tool. It can steal everything from the keys you are pressing to what is on your screen to what is being said near the machine.

It also has some very unusual data stealing features including reaching out to any Bluetooth enabled device nearby to see what it can steal.

Just like Stuxnet, this malware can spread by USB stick, i.e. it doesn’t need to be connected to a network, although it has that capability as well.

This wasn’t written by some spotty teenager in his/her bedroom. It is large, complicated and dedicated to stealing data whilst remaining hidden for a long time.

Some features may be less used or unusual, but I really don’t see it any more advanced than other things. In fact, dare I say its in this day and age rather the opposite. Pretty much all the features are versions of stuff done a long time ago, so why not seeing it sooner? I won’t comment on the programming of it, as I’ve not seen the source, and the person or people involved obviously put a lot of effort in to it (which is to be commended), and it did hide itself for some while, too, which is also interesting. On the last line of the quote though, another person in the article wrote this (and they’re related):

Currently there are three known classes of players who develop malware and spyware: hacktivists, cybercriminals and nation states.

Nonsense. Complete and utter nonsense. To me, a cyber criminal is someone who for instance, steals money or someones identity via technology. Many years ago, I actually knew quite a few virus writers. I didn’t approve of some things (e.g., tricking unsuspecting victims into activating CIH virus [which had some interesting properties, too]), and never was fond of malicious code, but they were an interesting bunch. And a lot (more often than not, actually) didn’t have destructive payloads. As for those involved, some were actually quite talented assembly (and other languages) programmers (which is what drew me to them). They all had different backgrounds, different goals, and came from different countries. But one thing is certain: they weren’t state (They were often fearful of such), they certainly didn’t steal money or similar, and they were not hacktivists, nor anything related to the common usage of the word ‘hacker’. Further, some were teenagers, and some were in it not for harming others or gain, but for learning. Yes, believe it or not, you can learn a lot by studying assembly language or in general source code.

The whole point though: not everything that is new and unique is the most sophisticated thing. If you say that every time, it is basically a version of the ‘boy who cried wolf’. That is a huge problem for security. It is also sensationalism, and besides the media and government, who likes that? It doesn’t help anyone with their computers, not one bit. If a security company’s employees always say this stuff, and don’t even help people (besides telling them to buy their software), then what are the true intentions and who is really being helped? That’s the problem with spreading fear, uncertainty and doubt.

The Enumeration Casting Problem

The C and C++ keyword enum is a useful feature in the language(s). What it allows, is, to be blunt, an enumeration. A common example is, a group of constants, that are related. For example, you might have an enum of colors – something like :

enum Color {
RED,
GREEN,
BLUE
};

By default, the first value is 0, the next is the previous value + 1. However, there’s some variations. First, you can tell the compiler the value of specific constants. For instance, if you want RED to start at 1, then you would do :

enum Color {
RED = 1,
GREEN,
BLUE
};

You can also declare something to be of type Color (after the enum Color has been defined; though see later about enum classes [feature of C++11]). There’s a common issue though, which I’ll get to in a bit. Firstly, know that there is also the anonymous enum. That generally means something without a name (clever name, I’m sure). What does it look like?

enum {
RED,
GREEN,
BLUE
};

And that concept is useful (though not required) in solving the issue I referred to. More on that later.

Now, in all the above cases, you can refer to the constant (by name, e.g., GREEN) by simply ‘GREEN’. Problem however, is if you have an enum of another type but also could use a name like GREEN. Now, that may not appear to be an issue, but it is. The reason is simple: the variable is of type Color, but that does not mean it is in scope Color. That is to say, it is very much like having two variables with the same name. For instance, an unsigned int called ‘i’ and an long int called ‘i’ in the same scope. The variables cannot be declared more than once by the same name in the same scope (you will get a redeclaration error).

So, C++11 added enum classes (and consequently, enum structs). Recall that a structure access modes are public by default and classes are not (hence the use of the friend declaration in C++). Otherwise, they are the same. So, how can you fix the name clashing?

You use enum classes! And just like a class/struct, in C++, you can also inherit (in the case of enums, as far as I recall, it is of integral types). That means that you can make an enum that inherits from long long (as opposed to the default int).

Here’s how :

enum class Color : long long {
RED = 1,
GREEN,
BLUE
};

However, how do you access it? It’s some what of a ‘static’ variable in a class. What that means, is, you specify the scope, in this case the scope ‘Color’. So, for example, to make a Color variable BLUE in the above enum, you would do this :

Color c = Color::BLUE;

Now, back to where I referred to a problem that is commonly encountered. One example, is, the switch statement, and in particular, case blocks. I won’t discuss that one, but when you do encounter it, you can get around it the same way as I will explain. It will basically be a type conversion error. Sure, you can add a cast, but that seems so wrong in a more type strict language. After all, this is C++, and not C – you don’t cast nearly as much, and ideally you don’t cast at all (obviously, that isn’t always possible, say socket handling via the BSD Socket APIs but its best to try to avoid it). Is there really any harm in it, for an enum? Not really for a basic, but it also isn’t necessary (in that case or enum classes).

The specific problem I had, is for example, an enum sort of like this :

enum class Flags {
DEBUG,
LOG
};

Now, although I could prefix DEBUG and LOG with something, I think it makes more sense to have intention fairly clear (and the names [not in this example] were relevant to multiple groups of flag types). So, if I had another enum, or some thing else with the name DEBUG or LOG, then I would have name clashes. So, my attempt to fix it (above) would allow me to specify Flags::DEBUG. However, when you make it a class, it is, well, a class. So, we’ll assume that we have a function called set_bit that takes one argument: an unsigned short. What happens if we try to pass a Flags::LOG to it? It won’t work, because its not actually an unsigned short. To that end, a ‘case’ statement in a switch block, of that type, won’t work either because switch/case expects integral types. So, what can be done to work around this?

Maybe there is a better way, and sure you could just have them be individual variables, but what I found as a nice way, is the following set up:

Use a namespace by the name Flags. Inside that namespace, you can put an anonymous enum (or not anonymous if you want) in with the variables. This will isolate the variable to namespace Flags.
Thus, you can now do :

namespace Flags {
enum {
DEBUG,
LOG
};
}

and indeed pass Flags::DEBUG or Flags::LOG to the set_bit function.

Only thing to be aware of, is that namespaces do have one feature that could be an issue. Its not really hard to fix, though, and issue is relatively speaking (loosely defined). Namespaces can be in multiple files; that is, you can define enums in the namespace in one file, and in another file do the same thing. Basically, though, if you have a risk of that, you can use the same idea. For instance, nesting namespaces (e.g., namespace somenamespace inside another, and inside somenamespace you have the enum).

The real benefit of this idea though, is that it allows you to make more use out of enums (when what you’re doing would make use of enums), without having to use casts or worry about name clashes. And of course, you can also just use a separate variables inside a namespace. It really is up to style and the actual reasons for needing the variables. In my case, I created a dynamically sized set of bits (read : std::bitset but not having to have the size at compile time. And no, it has nothing to do with a vector of bools) and I was tired of having code that looked like a file of #define’s (even though I didn’t use #define, and instead used const’s, the fact is it was individual variables and a long list with prefix’s to each variable). I didn’t like that. This is not C, and anyway C has const too nowadays. So, I wanted to clean things up, and in my case, this was quite useful.

Live Update: Fedora 16 -> Fedora 17

Although its still only beta, I was excited about a few significant changes in Fedora Core 17. I’ll explain.

Firstly, there is UsrMerge, which is explained in detail at http://www.freedesktop.org/wiki/Software/systemd/TheCaseForTheUsrMerge. I like the benefits of this, and it seems the best alternative to some other changes over time. Those changes:

Before UsrMerge, we had the following directories, among others :


/sbin/
/bin/
/usr/sbin/
/usr/bin

The original point of sbin versus bin has nothing to do with security and no, the ‘s’ does not stand for super or root user binaries only. It was static binaries. This was important in the days /usr and various other mount points were on separate file systems. Why? Static binary refers to the libraries it needs. Rather than load them dynamically, it was all built into the binary. This was vital when you had a problem on boot up and you needed certain utilities (if e.g., it needed something on /usr – what if that wasn’t mounted for some reason [whatever the case] ? You’d be in trouble). However, over time this changed. Nowadays, in systems prior to UsrMerge (e.g., CentOS), most of the files in /sbin are dynamically linked. In fact, in systemd setups (as opposed to the older init scripts), the /usr isn’t even supposed to be on a separate file system (when I first read that it annoyed me, but I’m ok with it now).

So basically, what UsrMerge does is the following :

Move all files in /sbin, /bin, /lib, /lib64 to to /usr which essentially means move the files, not the directories. After that, whatever was in /sbin is in /usr/sbin, and the same applies to /bin, /lib and /lib64. But what next? It symlinks (e.g., ln -s) the old locations to the new. So, essentially, when you do an ls -l / you will see something like:


lrwxrwxrwx 1 root root 7 Apr 20 13:13 bin -> usr/bin
lrwxrwxrwx 1 root root 7 Apr 20 13:13 lib -> usr/lib
lrwxrwxrwx 1 root root 9 Apr 20 13:13 lib64 -> usr/lib64
lrwxrwxrwx 1 root root 8 Apr 20 13:13 sbin -> usr/sbin

This only has benefits.

The other change I was looking forward to – the default gcc and gcc-c++ packages are now at version 4.7.0. Note I had only one minor problem when updating the operating system. A mis-sight on my part with my libstdc++47 package I built. The problem was file clashing. The solution is simply to remove that package. What I should have done is had /usr/include/c++/4.7.0 in a different directory name. As it is, I only need it on a CentOS 5 machine, which equates to never having libstdc++ 4.7.0, so I”m not concerned about fixing it (especially with such a simple fix – vital especially if you were to upgrade versus fresh install by DVD. The file clashes would stop the installer and then your system could be hosed. So do keep that in mind, folks). Normally I reinstall fresh, but I didn’t want that for a variety of reasons. In the end it was well worth it.

There’s a few addendums I’ll add though. Firstly, the instructions of what I followed are at https://fedoraproject.org/wiki/Upgrading_Fedora_using_yum#Fedora_16_-.3E_Fedora_17.

Note though that I had already downloaded the beta DVD as I intended to go that way initially (I downloaded it hours before the official beta was announced; little known fact that its usually obtainable by such time in at least some mirrors). And being on my rather lower downstream link, I did not want to download 1.6GB (which is what yum said I’d have to download). Now, how could I fix that? Well, because I had the DVD, I loop mounted it (see man page for mount, otherwise its : mount -o loop /path/to/image /mountpoint). Then, I initially was going to create a repo but (likely due to me being very impatient and not setting it up right) yum did not want to download via the loop. So, better idea came to mind:

What if I cp the files from the loop-mounted iso image to the location yum downloads the files it needs to? In this case it is /var/cache/yum/x86_64/17/fedora/packages/ (I actually didn’t do cp but rather an rsync as I had downloaded a few there already). As you can probably guess, it means any file that is the same version/release (and all that good stuff) will not have to be redownloaded. That means in my case instead of downloading 1.6GB I had to only download < 900GB. Quite a nice improvement. The only other note that comes to mind: on shutdown, after the update, the kernel could possibly panic (it did in my case after trying [and maybe did succeed - can't recall] - umount the file systems. Since I had no file system issues on power off and then power on (and boot), I assume it did succeed to umount things properly. This may seem like an isue but it's not really that shocking: all the libraries have been updated, kernel, the environment, everything. Next shutdown should go fine.

Aside from that, make sure you heed the warning about dracut and the kernel: it will assume by default you mean the currently RUNNING kernel. That means if you installed any kernel and have not booted into it, then you either must remove that kernel (by version and architecture!! That is essential.) or reboot into the new kernel before building your initramfs. I should point out another thing about initramfs: that's another reason why /sbin didn't care about libraries being statically linked or not - it has the modules/etc that you need to boot up with that kernel. Regardless though, UsrMerge works fine, and no bugs that I can see in Fedora Core 17. Of course, there is one issue for debugging with selinux enabled. However, you simply have to disable that flag (it's related to SELinux denies access to the ptrace system call). Simply doing 'setsebool deny_ptrace 0' will solve that (and if you use debuggers or programs such as strace, then you'll need that). Otherwise, see for more information.

The Internet Is Working – We Don’t Need to Start Over

Quick addendum (12:00:54 PST): This is no joke. I’m very serious when I said I read this on the BBC (sadly). For those curious, you can find the original article here: http://www.bbc.co.uk/news/technology-17032274

Indeed, this is going to be some what of a satire. Let me explain. The other day (a week or two ago actually), I saw a rather amusing article by a security expert (posted on the BBC). Now, I’m not going to say he is or isn’t an expert. We all have our strengths and weaknesses and that’s how humans are. But I will say that his thoughts mentioned in the article are completely ridiculous and outright incorrect, in many ways. The title of the article is ‘The Internet is Broken – we need to start over’. That’s why I reworded it.

The headline is actually a good way to start the article – it has a very good hook. So, what better way to start this, than by quoting it?

Last year, the level and ferocity of cyber-attacks on the internet reached such a horrendous level that some are now thinking the unthinkable: to let the internet wither on the vine and start up a new more robust one instead.

Maybe I’m a blind administrator, but I didn’t notice this. Oh, sure I noticed probes on Xexyl (and my other sites), and I noticed many companies being attacked successfully. What I did not notice however is this being anything new. Please, Professor Alan Woodward, could you tell me how this is any different from the other years? Just because they aren’t reported, does not mean the attacks didn’t happen. A lot of organizations, companies, whatever else, do not admit to attacks. Others may have been attacked but the attack did not succeed, either at all or enough. Further, many don’t even know it! (I have made several web hosting companies and even a technical school aware of the fact they had a compromised machine or network. It wasn’t hard to figure out when they started probing me, and they are a legit company. But they didn’t notice it. Wonder why that might not be reported then?).

Let’s go back in time, as I think it’s really important. Let’s go back to, say, November 2 of 1988. Why? The Robert T. Morris Worm would be pretty significant for that time and is a prime example of a simple truth: if you can make it then you can break it. That means, anything made by a human can be broken. Not only did the Morris worm exploit many different services and machines, it impacted them in a rather large way. They crawled to their knees. And what services did it impact on those machines? Some do come to mind immediately, but I’ll get the list from a source, so you get the full idea:

sendmail, finger, rsh/rexec and – this one is important – the weakest link in the chain: passwords, in particular weak (and the reason its the weakest is it’s a HUMAN creation). Further, due to the way the worm worked, it acted as a fork bomb, hence making them crawl to their knees. Only someone who is completely oblivious to the Morris worm would not think that’s a significant part of the Internet history. And when is that again? 1988. When was this article (the BBC one) I mentioned written in? 2012! And that is only ONE example. What about the CIH virus? That even was spread by accident on software from major retailers. And since it trashed the CMOS, it prevented machines from not booting at all until the board or at least chip was replaced. What about when the 13 year old from Canada, known as Mafiaboy took down eBay, Amazon, Yahoo and several others websites by way of DDoS attacks (distributed denial of service attacks) years ago? I might add that a some of the attacks last year and the more recent years were DDoS attacks (certainly more than that, but they were still one type of the attacks). The point is the same, however: attacks on human creations – be it vandalism (graffiti, physically damaging someone’s car, whatever else), or a computer network, even a network of network – the Internet. They happen and they always will!

Now, there is something more important to realize is wrong or missing from this BBC article.

However, recently the evidence suggests that our efforts to secure the internet are becoming less and less effective, and so the idea of a radical alternative suddenly starts to look less laughable.

The only thing that is laughable, is that suggestion. Are you really considering throwing out 40 some years of many many people’s creations and work? Seriously? That is frankly disgraceful and shows how destructive humans are! You claim security breaches cause lost revenue. That’s true. So does shoplifting. So does a global economic crisis. So do many other things. The fact is, the ways to protect is not the problem. The problem is people have always been short sighted in the planning stage of things, and furthermore, lazy/ignorant of what is necessary – implementing a policy, having the proper skill set, and so on. Where do you think the quote hindsight is perfect vision (and variants) comes from?

No matter what, peoples creations are bound to be flawed in some way, at some time. That’s how this world is. To throw out 40+ years of development because some like to cause trouble would also cause a lot of money loss. Firstly, some companies are only online businesses. So to take the ‘Internet’ offline would put them out of business (and even if they moved to a local store, it means they’d have to either build additional stores = more money spent, or they’d have a far smaller customer base). And surely you understand supply and demand, so when these companies go out of business, prices will potentially rise for those who do survive.

The fact of the matter is this: developing something, you are bound to make a mess. That’s expected. It also doesn’t matter. What does matter, is how you react and address the issues that come up. And, that’s exactly what has been done. For example, I mentioned rsh/rexec. I remember when these were more common, and the flaws they had were hard to believe. But they were still there! You know what is far more common now? SSH (which allows for a remote shell and also running commands remotely). You also have scp (secure shell’s remote copy). You don’t see telnet services as much either, do you? That’s because it is less secure than say, ssh.

No matter what, nothing is perfect in this world. Nothing. Trying to be perfect is an easy way to go absolutely bonkers.  If you ask anyone (including myself, I admit) how perfectionism impacts their life, if they have it bad enough, you’ll at least hear or see what I mean.

To summarize, a good friend from Holland once told me about a saying there. When translated to English, it means this :

Where there is lumber work, there’s wood chips.

I’ve since then have passed that on to other people, when for example, they made a mistake that had them down. That saying is golden. It’s absolutely true, and its something everyone can think of at some point or points in their life. After all, no one is perfect….

(As an aside: all the above considered, the internet is pointless if it doesn’t exist, and it would take some long time to be back up. What Alan Woodward is suggestion reminds me of security through obscurity. A perfect Internet in a non perfect world is impossible. That’s the bottom line.)

C++11: std::unique_ptr, raw pointers, and containers

Last Updated on 2012/07/13.

Since I now have my own RPM repository on two servers each of 100Mbps, I’ve removed the /rpmbuild directory here on my server. That means you no longer need to build the backports of GCC 4.7.0 yourself but can instead just use the Xexyl RPM repository. I documented it here. Now, that that’s cleared up, it’s been a while that GCC 4.7.0 is out. Here is the updated article, along with a mistake I made fixed:

I’ll say that GCC 4.7.0 is more restrictive and THAT is NOT BAD but actually GOOD. It not only will tell you about more possible issues in your code, it can reveal potential bugs in your program (that end up being compile time errors or warnings in other cases). That is good. Never ignore warnings (obviously errors you can’t). Okay, you can ignore them when say compiling a huge program (like GCC), but those are tested thoroughly and that even runs a test suite after compiling. The reason you don’t want to ignore them at compile time is because if they crop up later in the runtime, then you may be hard pressed to know where the real problem is (I’ve discussed this before: see my article about memory corruption).

Among one of the new features of C++11 is the major improvement to smart pointers. The unique_ptr recognize and accepts move semantics but it does not accept or recognize copy semantics. Some constructors will automatically be default deleted because of this. But that’s not bad, if you think about it: if you have unique ownership, then you aren’t very likely to have one ‘owner’ delete (as opposed to C’s malloc and free functions) it, only to use it later and dump core (and that would be the best option, as I discussed before in the other article).

I’m not about to begin explaining move semantics or perfect forwarding: there’s enough documents out there about it, including the original proposal/draft. Instead, I’ll discuss the smart pointer known as unique_ptr (in namespace std), and unique_ptr’s in standard containers (maps, multimaps, their unordered counterparts [also new feature of C++11], and vectors).

A word about raw pointers. Since unique_ptr’s only allow one owner, you have do have to take care of the fact that you can move it to a new owner, or not. But, what if you want to, say, keep its owner (say, have it in a map or some container). You want the container to be the owner. But then you want to (or need) to modify something in a function or class function. Or what if you need the pointer in more than one location? For example, in the project I referred to in my latest post (before this one), it has a map of ‘characters’ (it’s a rewrite of a type of game that is the predecessor to today’s MMORPGS). This list/map/whatever owns the pointers, and when it is removed from the map, its destructor is called. However, it would be less efficient (certainly more time consuming for the engine) to go through (potentially) the entire list (say, over 2000) just to find a character or item in a location that is already known (we’ll call it a ‘room’, e.g., to look at a character there). In short, it is some times convenient to have a list (e.g., a vector) of raw pointers. The class that has this list does not necessarily own the memory, nor does it have to allocate, or do anything except remove and add the pointer to a list. When its removed from the object, the real piece of memory still exists, just the object no longer has direct access.

So, anyone who has worked with smart pointers would know there is a way to get the raw pointer. A smart pointer has the get() member function. Now, note that the variable in question, is not (for example) a Object*. Instead, it is std::unique_ptr<Object>. What does this have to do with anything? It means as much as this. Normally you  would dereference the pointer to a class or structure, by, say the -> operator, e.g., character->get(). However, it isn’t the way in this case. Remember that the variable is not a pointer. Rather, its std::unique_ptr holding a pointer to an instance of an object (say of type Object). In other words, it holds an Object*. That means we don’t use operator-> but instead the dot operator. What this basically means, is we would do something like this (if we had an unique_ptr and we wanted a raw pointer to pass to some function) :

Object* o = uptr.get();

After that, uptr is the unique_ptr (and still has ownership), o is the raw pointer, and *o is the actual object (recall that * is the dereference operator in this context and to dereference a pointer means to access the actual place in memory that the pointer points to).

Does this change when its a container of std::unique_ptr’s? If you have an iterator for example, then absolutely yes. Why? Well, before you have the actual std::unique_ptr, right? It’s not actually a pointer itself, so you use the dot operator. But if you’re iterating through a vector (for example) then you would use operator-> ( or the equilvalent way: (*iter).member ). This is because again the iterator is not the actual object but (essentially, certainly if you think in terms of functionality) a pointer. And what happens when you want to access a member of a pointer to some structure or class? You dereference it first. To add to that, and possibly confuse matters, if it’s a tuple (of some kind) then you have the first and second members. So if you have a std::pair<int, std::unique_ptr<Object> > called mypair, then mypair.first is an int, and mypair.second is an unique_ptr. So if you’re iterating through a container that holds that pair (say, a map) then you access it more like this :

iter->second.get()

or

(*iter).second.get()

Basically, iter is the iterator, you dereference iter and access the member second (which is in this case an unique_ptr) and then you access the get function on that object. That function will return the raw pointer.

There’s two other new smart pointer types: std::shared_ptr (think of reference counting) and std::weak_ptr (which is used along with shared_ptr’s). There’s many other nice additions. That includes threading support, new usage of auto keyword (yes, folks, it existed for decades despite what some incorrectly think), a new way to work with time (system time, durations, etc.) and much much more. I would suggest you get used to the new standard if you use C++, because it has a lot of nice additions.

To close, I’ll go back to something I said I won’t go into detail with – the move semantics. One thing I will show is an example used in the proposal. Due to variadic templates, we have perfect forwarding. This allows something really cool. Let’s say you don’t know how many arguments will be passed to a constructor. Then, say, you want to create a lot of unique_ptr’s. Well, you could write out the more explicit stuff, or you can do what I do to make code clean. That is to say, have a header file that includes the libstdc++ header file that makes std::unique_ptr (and other things) available and add a function that does perfect forwarding to the constructor. The file is simply this :

#ifndef __MEMORY__
#define __MEMORY__
#include <memory>
template<typename T, typename …Args>
std::unique_ptr<T>
make_unique(Args&&… args)
{
return std::unique_ptr<T>(new T(std::forward<Args>(args)…));
};
#endif /* __MEMORY__ */

To use, simply do something like :

auto obj = make_unique<Type>(arg1, argN, ...);

Yes, that’s what auto now does: it automatically deduces the type as best it can. If it can’t, the compiler will tell you so and you’ll have to fix it.

Backport GCC 4.6.2 to CentOS 5.x

NOTE: This post is mostly obsolete because I have my own RPM repository now. So, while the instructions are still below, I have since removed the files on my server, as it is both a waste of space and gcc4.6.2 is not the current version any more anyway. You can – if you want to see the spec file, just install the xexyl-release file (that I described elsewhere) and use ‘yumdownloader –source gcc47′. After that downloads, you can extract it and inspect it, rebuild, or whatever else. So, yes, you will get a 404 not found if you run some of the commands below, because the files aren’t there (and that’s what prompted me to write this note, actually).

(Update 2012/March/19: I confirmed and fixed some information on the mock setup portion of the commands below – regarding the second to the last command in that part)

The past year or so I’ve been working on a complete rewrite of an old project of mine. While it wasn’t originally mine, it basically is up to me these days for updates. Though my friend who started it is still around and knows about this rewrite, he’s very busy lately with his own company and other things.

There is (was) a problem with my rewrite though. Not the rewrite itself, however. It’s more that it uses C++ instead of C. However, with that I develop under Fedora Core 16. That’s not a problem itself, except its more up to date. That means it has more of the new C++ standard that was recently made official. Known as the C++11 standard (11 for 2011 – the year it was made standard) it is very much an improvement over the older (now obsoleted standard). However, the server we run the project on (it happens to be an older type of game – the predecessors to the MMORPG’s – it’s known as a multi user dungeon, or mud) is based on CentOS 5.x. And even if we upgraded to CentOS 6.x, that’s still too far behind for the newer features.

There’s always going to be complaints of library dependencies. In the Windows world this was called DLL hell. My understanding is that is still a terrible problem. However, in Linux we have so files (shared objects) – essentially libraries that aren’t linked in statically to the binary – they are loaded when needed if they can be found (else you have other issues). Now, one might think in CentOS (or any Linux/Unix) you’re therefore just as out of luck as the others.  But not so. However, first, there’s another issue :

If you’re using a binary distribution, then the programs are already compiled and therefore linked. So, when you try to install an updated package (e.g., a  RPM for Fedora Core) under a different system, it’s going to have issues (dependencies). So, then you think “I’ll just update those other packages.” Well, actually, you won’t. Not if you don’t want a broken system that is. Ok, you _could_ use rpm options –nodeps while installing it, but that is an absolutely terrible idea in this case (it might even fail for all I know). If you did this and it ‘succeeded’, you’ll at best be unable to run a lot of programs.

So what can you do then ? What did I do to fix my issue if these are so true ? The beauty of open source comes to the rescue. What if I actually compile the program myself and link it to the libraries that are on the system already? Some might reply back that there’s also a problem with that : you either have to store it in /usr/local/ or you have to risk clashing files. Well, its true that is partly correct. However, there’s other options. Much like GCC4.4.x was backported to CentOS 5.x (eg, the packages that start with gcc44), you can also backport GCC 4.6.2 _and_ the libraries. RedHat already did back port 4.4 (hence the suggestion above). I went further and backported 4.6.2 and the new libraries.

Before I show how though, I’d like to answer ‘what is a backport ?’. Simply put, CentOS (and other distributions) will backport security updates. It basically means: we take updates and merge them into the old(er) versions of the program. Therefore, we don’t update it entirely (and thus risk stability and every thing else, e.g., having newer libraries to worry about) – we just  build it against the libraries we have for that system.

That’s exactly what I did. Actually, I did do it under a mock chroot (mock is an updated mach = make a chroot). I won’t explain too much of the RPM spec file, but I will link to it. Note that if I were to include the source rpm file you’d be downloading it for some long while; therefore you will have to generate the tarballs of the files in it. The rest – patches – I will throw into a zip file and include that here. So, here’s how to do it under CentOS (and also note that I only build C, C++ and disable profiling [I didn’t have java installed and didn’t need it either, and profiling had some other issues [at least originally; additional changes I added may have fixed that but I haven’t tested it as I don’t need it and it is after all a backport and not the official package). Note also that it will NOT clash file names. All files that are the same, e.g., gcc, g++ have a suffix: 46 (therefore its invoked when you type in gcc46 or g++46 for instance). I also use have the libraries provide the other version, so updates should not make a problem. You could also just include statics libraries under the proper directory (which I do for libstdc++ for example) and include an ld script so that it uses that instead).

For building, I suggest you follow these commands. It will install mock, set it up and then chroot into it. Then it’ll generate the right files and start the build.

First, in the host system do the following as root :

  • yum install mock
  • usermod –append -G mock username
  • su – username
  • mock -v –init
  • mock –install yum
  • mock –shell

I previously noted that the second to last one might not be necessary. However, I just confirmed it is necessary for later, say, when you want to install some package (e.g., the ones you create). Therefore, the instructions should be correct in all cases. In addition, I realized one other thing (I mentioned it in another post related to backports, but I never updated this until now, 14 April of 2012): you may have to install svn first – I added the command to the list below as of today.

Now, you should be in a mock chroot. Do the following :

  • yum install svn
  • cd /builddir/build/SOURCES
  • svn export svn://gcc.gnu.org/svn/gcc/branches/redhat/gcc-4_6-branch@180561
    gcc-4.6.2-20111027
  • tar cf – gcc-4.6.2-20111027 | bzip2 -9 > gcc-4.6.2-20111027.tar.bz2
  • wget http://www.mpfr.org/mpfr-2.4.2/mpfr-2.4.2.tar.bz2
  • wget ftp://gcc.gnu.org/pub/gcc/infrastructure/gmp-4.3.2.tar.bz2
  • cd ..
  • wget http://xexyl.net/rpmbuild/gcc46.tar.bz2 && tar xvf gcc46.tar.bz2
  • rpmbuild -ba SPECS/gcc46.spec

If you have everything in place it should build what you need. You can then install the resulting RPMS that are under ‘/builddir/build/RPMS’.

That’s all there is to it if every thing goes well (if it doesn’t, I’m particularly bad about mentioning ways to get in touch with me but if I do find anything wrong I’ll update it).

Oh, and it goes without saying (but saying it just to be clear) I won’t be responsible if you  do something different or something goes wrong. We all take in information but its up to oneself of how you use the information. I suggest building and installing in a chroot for a reason – to be sure things do not go wrong. It works fine here but you should not take that for granted and just assume you shouldn’t be cautious.

Bring Your Own Demon

The other day I saw reference to something abbreviated as ‘BYOD’. They claim it stands for ‘Bring Your Own Device’ (e.g., to work). I say that’s more like bringing a demon to your office and I mean demon in the most thought of sense: pure hell.

I especially love this part of the article mentioning it :

Out with the old: You may find yourself using your own device – laptop, tablet and/or smartphone – for work whether you like it or not.

The only thing that’s ‘out’ is the author (from BBC’s) mind and I mean that in the sense of out of touch with reality (of security, business perspective, liabilities, etc.). This idea is about as dumb as it gets with current standards. Allow me to explain this in a different way, for the non system and network administrators….

As an admin of a very small network (we’re talking < 10 devices connected), I would dread this idea. I cannot imagine this for a COMPANY – even medium size or small, let alone a large company. What a bloody nightmare that would be! You heard of the bastard operator from hell? Well, if not, let’s just say you would if this came to any administrator with half an ounce of sense (and I’m sorry, but that’s about as nice as I can muster it up).

The very fact of the matter is, this is a nice way to get your network breached. I guess they’re forgetting the old type of virus called a master boot record / boot sector infecter ? All it took is someone unknowingly putting a floppy in the computer (that was infected) and then the computer shuts down.. or is rebooted (if up). Or maybe its put in (without thinking more so) and then the computer turned on (e.g., a workstation). Monkey brain anyone? Yes, there’s word play involved: on one hand, I am describing the administrator (that wants to allow employees to bring their own devices that is), and on the other its a reference to the Monkey and Brain viruses, both of which were MBR/BS viruses.

And if that’s bad enough, maybe you need to brush up on your security terms (yes, just knowing OF these things along with some BASIC logic would be enough to concern any intelligent administrator). I’ll mention some of them for you.

  • Worm – a program similar to a computer virus, only it has the added ability to spread by network (this could be through a flaw/hole in a service, an operating system, or even something as simple as emailing it with a convincing message that running the program is a good idea [or exploiting email clients that run executables or scripts]). You think the virus is bad? What about a worm? What if it exported data? Oh, and see next term.
  • Backdoor – a program which ‘opens a door’ (e.g., some port on  the victim computer) that allows  access to the backdoor and potentially the entire system. Cleverly named, isn’t it?
  • Denial of Service, Distributed Denial of Service, etc. I don’t think I have to discuss this one much.
  • Spam botnet anyone?

The list could go on for quite a while. And to make it worse, you have additional ideas that come out all the time. I remember years ago – someone asked on a forum I was part of, if it was possible for an image to to contain executable code. I replied with a resounding ‘yes’ – under a condition. The condition? An image viewer being stupid enough to actually interpret/execute code found IN the image! What happened some months later? I seem to remember it actually happened as such. I know the first part is 100% valid, but I’m pretty sure a certain software company in Redmond, Washington did it. Great!

I’m sorry to say this, but allowing employees to bring their own device to a company’s network and expecting it to work is about as stupid as giving an unexploded grenade to a 2 year old to play with (or similar – a loaded firearm of some kind). It’s really that simple. It’s opening a huge can of ugly worms (possibly literally) and you may as well expose the ENTIRE CORPORATE NETWORK to the outside. Forget all the layers of security – the DMZ, the firewall(s). Just forget all that stuff is in place! In short, let’s all bring our own demons to work! If it isn’t broken, we’ll break it so they can fix it (and we’ll also whine when they can’t fix it as quickly as we like).

I’d hope that most companies see the stupidity in allowing such a thing. I know some major corporations do see that. That’s why they prevent software being installed by employees and have other protections and even company policies (“you’re fired if X, Y, Z”) that specifically do NOT allow this type of stuff.

I admit I’d not mind being able to have my own choice of OS BUT for EVERY employee (e.g., corporate policy expects you to) to bring in their own device – without any kind of audit especially – is stupid. The fact is you cannot expect things to go OK when this isn’t the case (as in you’re not allowed to bring your own device) so how in the world they think they can keep control when you’re allowed to is well beyond me… (mainly because it’d be a bloody nightmare and simply impossible with how things work). I’d also like to say that bringing a company laptop is different.

But in short, you’ll have issues come up. And if you allow the BYOD then you’re going to have even more come up, guaranteed (you don’t know where that was!).

The Power of 8

So, today I turned 30 years old. Until I wrote this (which was two or three days ago) I wasn’t really sure how happy I was about it. However, as writing is considered therapeutic, I not only wrote a guide on numbers (and different number bases e.g., binary [2], decimal [10], and hexadecimal [16]) I also added some fun ways of looking at age in general, and indeed how so many things can be looked at in far more ways than seems possible. The quickly wrote this but it should be clear if you follow the examples. In any case, here’s how to convert between decimal, binary and hexadecimal. It also indeed explains what I mean when I say ‘The Power of 8′. So what makes 8 so powerful?

If you read what is below, you’ll know (at least to some degree). You may know already, but in any case, here’s a more mathematics article from me and something I never thought I’d write about (whether its a precursor to something more advanced I do not know) …

So, firstly, what is so powerful about 8 ? Well, quite a lot. “But, it’s a single digit!” you fire back. Ah, but is it REALLY a single digit? Doesn’t it really depend on interpretation? “Yes, but so does everything in this world!”. Indeed, and that includes numbers. In fact, if you read on, you may realize how 8 is very powerful indeed. So powerful that as you read this, 8 has already crossed your path and gone into the very device you’re reading this from… (Yes, that’s a mere analogy or even a metaphor of a concept and type of data known as a ‘byte’).

To get to the bottom of this, you have to understand that what appears to be a single number isn’t a single number all by itself. Even 10 is a single number yet when you think about it, its composed of two digits; there is no digit “10″. That’s two digits which is each composed of something more hidden – a bit. In actuality, a number is a combination of bits. Remember math class ? You would study the 1s places, the 10s, the 100s, and so on.

The thing is, ALL numbers can be looked at this way. So what’s so special about 8? Although it depends on other factors, its generally considered that 8 bits is equal to 1 byte (this can also be 7 bits). What is also true is that one BYTE can hold more than 1 ‘number’. Indeed, if you take 8 bits (1 byte), you can count from 0 to 255 or even
-128 to 127 (if signed).

To realize how, you simply have to realize that a single bit is one of two numbers – it can be 1 or it can be 0. If you take that into account, and you also take your mathematics class into consideration, you’ll realize that if its a ’1′ in that position or place, you add it and if it’s a ’0′ then you do not add it. That means in decimal (base 10), you would have the following :

 100
+ 10
+  1
----

How do you solve this? You can say you take each value and add it up. But there’s more to it, isn’t there? What you’re really doing is lower level stuff – it’s closer to bits than you might realize. In the ones column, you can say you have (in this example) 1 * (10 ^ 0).

Why to the power of 0 ? Well, what if you did not start with 0 ? 0 comes before 1 which comes before 2, and so on, all the way to 9. If you did not have 0 you could not have 10. Therefore we start at 0. So, understand (remember) anything to the power of 0 is simply 1 (except 0 which is naturally 0).

Important part to understand is, each time you move over to the next bit you use to the power of that bit (position). Or to put it another way, you have in the above problem :

(1 * (10 ^ 2)) + (1 * (10 ^ 1)) + (1 * (10 ^ 0))

If you take a calculator that understands the order of operations (I mention this as I hear that in some modes in some Windows versions their calculator does not!) you will get the following :

100 + 10 + 1

And we all can conclude from that that the result is what it should be : 111, or one hundred, one ten, one one.

What about this though ?

 250
+ 90
+  1
----

Is it really the same idea? Not only is the answer ‘yes’, its even easier (in some ways)
than what you might remember from maths class. In that class, you might remember carry-overs? As in, if the column’s values add up to 10 or greater, you will need to carry the >=10 over to the next column. That’s fine for the basic class, but what the end result is, when you evaluate what I’m writing about, is that its really this (realize that there’s not only one single way to solve a problem that has multiple steps and even one step problems could be solved different ways in many things) :

(2 * (10 ^ 2)) + ((5 + 9) * (10 ^ 1)) + (1 * (10 ^ 0))

… which expands to the following :

(2 * 100) + (14 * 10) + (1 * 1)

… which expands down to :

200 + 140 + 1 = 341

So, that’s decimal in a nutshell. What about base 2 (i.e., two digits : 0 or 1 aka binary) ?

It is actually the same idea, only you deal with 1 or 0, and instead of 10  ^ pos you have 2 ^ pos.

So what is 15 in binary ?

Well, since 15 is most certainly >= 0 and < 255, that means it does indeed fit in <= 8 bits.

That means that we actually only need 1 (or not even 1) byte (point : indeed, numbers can indeed take more than one byte but 15 is not such a number).

So, think about it for a minute. We have either a 0 or 1 per place. If it’s 1, you can consider it as ‘set’ or ‘true’ or ‘on’. If 0, then ‘not set’, ‘false’, or ‘off’. This is a rather useful thing to keep in mind as it makes converting to binary from decimal or even from binary to decimal rather simple.  If you realize that we’re discussing base 2, then it’s multiples of 2. What that means is simply the 0th position (1s place) is 1 (if it is set). The 1st position (tens place) is 2 (if it is set), the 2nd (hundreds place) is 4 (if set). That makes it very easy to think of it in those terms, for many calculations.

If you look at it that way, you can easily convert by simple addition. Indeed, 15 is actually 1 + 2 + 4 + 8 which equates to what in binary ?

00001111

(and remember that the leading zeros are not even really required though it does give some way to realize its binary if the base is not specified else where).

And that is why 8 is so powerful: it can hold 256 different numbers, and it also leads into the inspiration of writing this whole piece. I turned 30 today (14/Feb/2012). I was for various reasons – as I think is fairly common among a lot of people when they get close to it – not very happy about it. I wanted to come up with some nicer way to look at it, and at the same time write something technical.

But along with the technical aspect, comes some fun. Because you can interpret things in so many ways, you can make the 30 different depending on base (or even just interpretation). There’s even an old quote (that I read years and years ago) that is something like :

“I will tell you my age but I won’t tell you which base its in.”

So, along those lines, here’s different ways of interpreting the number 30 :

  • 30 is equal to 4 0s followed by 4 1s.
  • 30 can be represented with base 2 (any number can be, even negatives [a different topic entirely, and more technical]) but yet if you divide it by 2 you get 15 (which cannot be equally divided by 2 without getting a fraction) – or its not a multiple of 2.
  • If you consider another base – base 16, which is to say 0 -9, A = 10, B = 11, … F = 15 (= base 16), then I could say I’m 1E.  The way to determine it is really simple: break it into sets of 4 bits and do the binary conversion. And if you understand it enough, you can do it in your head (that’s actually the inspiration for this article in part; I wondered what my coming up age would be in hexadecimal, and then I did it [away from any mathematical device] and thought it could be fun to write about).

Anyway, an example :

E is equal to 14. That’s equal to 1110 in binary. That’s the first set. Then, 1 is 0001. Put them together and you have 11110 which means 16 + 8 + 4 + 2 + 0, which is 30. So to get from 30 to hexadecimal, you can break it down into a combination of numbers (of multiples of 2, ie. 1, 2, 4, 8, 16, 32 …). Once you have those, you convert it to binary (or skip it if you can). Then each set of 4 bits can be converted to hexadecimal.
Question: why 4 bits? It’s simple: 0-15 can be represented in exactly 4 bits. 4 1s in binary is 15. It is that simple.

  • With respect to 1E – I could just be E years old (if you consider that is “one” E).
  • And what is an E? Is it an eternity? No but you could call it that I suppose.
  • 30 is represented in 5 bits, so I could be considered much younger than I thought (or maybe not!).
  • Another interesting thing: E in hex is 14 (decimal). And 1E = 30.
  • You could also call it 114, as it is “one” 14.
  • When I turn 60, I’ll be 3C. That means what? 3 Centuries old? I hope not! But you could call it that. You could also say my body temperature is 3 celcius (I must be cold blooded…). Then again, maybe not.
  • If I made it to 144 I would be 90 (in hexadecimal). The interesting thing there is if I actually did make it to 144d/90h, then I’d likely be doing nothing. Well guess what 90h in a computer (intel architecture eg x86 and x86_64) means? Nop whichis short for no operation. Interesting? Maybe, maybe not but it amuses me.

Essentially, the points of this article is that :

  1. You can look at things so many ways, and age is one of those things. You can be positive, you can be negative, but it’s healthier to be positive as you can be. EVERY THING in this world could be worse but you cannot change some things, so you should really only worry about what you can change (and actually do change – hopefully for the good).
  2. It also was to explain how numbers in computers work. Sure, I didn’t go into bitwise operations (e.g., bit shifting, twiddling, bitwise ANDs, OR’s, XOR’s, etc.) but I at least (hopefully) made it easier to see how the computer (and indeed humans) can see something one way or another when its entirely possible that there’s many other ways. In fact, computers typically use hexadecimal (it’s a multiple of 2 so its easy enough).

And that’s how numbers work in their most simple form. You can of course have negative numbers and even in binary, but as I think I mentioned that is more complicated and is more suited for an article by itself (which may or may not happen in the future by me).

In Honour of Charles Dickens

The other day I realized it was soon to be Charles Dickens’ 200th birthday. I was about to write something earlier about him and then got distracted. Well, now – since today it is his 200th birthday – I am writing something.

To me, Charles Dickens brings great pleasure through his literary works. Some might hear the name and immediately think of The Christmas Carol or Oliver Twist… and perhaps many others… but for me I think of the first work I read by him: Great Expectations.

It was required reading in high school (or was about ~15 years ago) and I was (as often was the case) the only one who really liked it. I thought it was a wonderful story.

Now, in the case of  Charles Dickens, I think his books are of the nature of you either really like them or you really don’t. Sure, there might be some of his books that aren’t this way, or there might be some cases where a person only likes some of a story by his, but in general I have heard various things about Dickens’ books :

  • It is too detailed.
  • Its a fantastic book.

That’s what I generally have seen or heard. And the first one even comes from someone who loves to read and has a very nice library, has a lot of good books to suggest and is the same person who introduced me to my favourite author: Jules Verne.

Still, I really enjoyed Charles Dickens. And while it has been years since I read any thing by him, I did want to take a moment to appreciate his books. He is – no matter what some people do or do not feel/believe/think – a fantastic author, a person who is well remembered for his books in different parts of the world. I can only assume he’ll always be remembered in this world, in some way or another.

And to those who likes the cartoon Bugs Bunny (and besides the politically correct people that influenced Warner Brothers to take some episodes off the air, who doesn’t?), a very interesting thing related to Dickens. Yes, there actually is the possibility of a link to a character in Bugs Bunny and Charles Dickens. Indeed, this is some speculation; however, it is quite possible nonetheless. To those who don’t know what I’m getting at, there’s a character in some of the Bugs Bunny episodes called ‘Elmer Fudd’. He’s actually is fairly small from memory, and so this could be even more likely. In any case, he’s a little humanoid that replaces r’s with w’s and tries to kill Bugs Bunny (who always outsmarts Fudd), Now, what does that have to do with Dickens? Read on…

To those who know the children story Tom Thumb, there was a person who was a real life dwarf whose stage name was General Tom Thumb (real name Charles Sherwood Stratton, he made fame working under P.T. Barnum). I learned of this last year. In any case, when I saw reference to General Tom Thumb, I found something (on Wikipedia) most interesting. It is a quote by General Tom Thumb and the explanation of it. Since I used to watch Bugs Bunny a lot and thoroughly enjoyed it (and even have or at least had VHS’s of cartoons with him in it), and I also have a liking to Charles Dickens, I realized the possible link immediately. Whether it is the real reason or not, I do not know. However, it is interesting to me nonetheless :

Waiter! bwing me a Welsh wabbit

The explorer John Palliser heard the above quote being said. I don’t know if it was Dickens who originally did this, but he at least popularized the mocking of the British aristocracy and the way they would order Welsh rarebit. And that mocking was done by substituting a ‘w’ for an ‘r’. So again, this may be speculation, or it may not be. I find it interesting nonetheless and I think his 200th birthday is the perfect time to suggest or mention it.

In any case, Charles Dickens will likely never be forgotten. I am thankful that I read some of his books, as I thoroughly enjoyed them. I still have plenty more of his to read, but that’s for another time, another day.

Authoritative Name Server (Ab)uses

Okay, so most network administrators and I suspect a fair amount of users know the main use of DNS zones and having authority. But to those who don’t know what it means, I’ll explain briefly:

It basically says this NS (name server) has control / is the master of the zone. That means, they can decide what the zone will resolve to, what sub domains in the zone will resolve to and so on.

Now, let’s also consider something very useful in BIND/Named (that’s the Berkeley Internet Name Domain / Name daemon). If you don’t know what a ‘daemon’ is: it’s a way for a computer to service a demon. Okay, maybe not. It’s actually a service – be it for web, dns, mail, list goes on. Apache (httpd) is a daemon and that’s what the d stands for in httpd too. It’s that simple.

Now, back to BIND. The feature I was referring to is known as ‘views’. It simply is this: you can match the source ip and say you use this definition of the zone’s file. This is very powerful indeed. It makes it possible to have internal hosts only visible on the inside. However, there’s indeed (as the title suggests) other – perhaps devious in ways – uses.

If you have authority of a zone, it means you can “play around” with clients. This has to be said though: its not normally a good idea, and its certainly not going to invite traffic if you try to exploit someone. However, when you check your daily logs and see a bunch of crap like this:

Feb  2 00:08:46 xexyl postfix/smtpd[16816]: warning: host-92-27-58-145.static.as13285.net[92.27.58.145]: SASL CRAM-MD5 authentication failed: PDI4NTc1ODM5NTkxMDkyNzkuMTMyODE3MDEyNEB4ZXh5bC5uZXQ+
Feb  2 00:09:20 xexyl postfix/smtpd[16816]: warning: host-92-27-58-145.static.as13285.net[92.27.58.145]: SASL CRAM-MD5 authentication failed: PDE5OTc2MDA2MjIyNjQ1MDQuMTMyODE3MDE0MkB4ZXh5bC5uZXQ+
Feb  2 00:10:09 xexyl postfix/smtpd[16816]: warning: host-92-27-58-145.static.as13285.net[92.27.58.145]: SASL CRAM-MD5 authentication failed: PDI2MTkwMzYzMDI1NjM2NDIuMTMyODE3MDE5MkB4ZXh5bC5uZXQ+
Feb  2 00:10:43 xexyl postfix/smtpd[16816]: warning: host-92-27-58-145.static.as13285.net[92.27.58.145]: SASL CRAM-MD5 authentication failed: PDc4NTkzMzEwMDUwMDU3NDYuMTMyODE3MDI0MUB4ZXh5bC5uZXQ+

… and you see that for several hours every minute… (Note: I had fail2ban active so it was blocking them and they would reconnect as soon as it expired as they were constantly trying to connect)

I would say its fair game. Incidentally, I had something similar recently (last year during the holidays). In that case I logged over 13,000 entries in the firewall (and that’s AFTER I added a ban from being annoyed enough. Though now I reinstalled fail2ban – so that’s less of a human intervention needed. But that’s another story anyway).

So, you see some stupid (likely Windows computer infected with the latest and greatest bug …) computer doing this, filling up log space, wasting resources and that includes network resources too. What can you do?

Well the first one (the other story) I was going to do similar and make it point to the broadcast of their network. It happened to be a Tech school (pathetic, yes, and even more so that they didn’t even acknowledge it like other schools, hosts and so on have – even after pointing it to them and calling them). That’s when it stopped.

However, today, I decided I only wanted to stop them. So, simply put. I set up a view for them. IF they are using your host name (and not IP) then here’s something for you to consider:

Match their IP in a view for them. Set up a zone file for your domain. Set all domains in that zone (as in, mail, www, ftp, whatever) MINUS the NS one (we want to make sure they see we’re authoritative) as localhost, that is to say: 127.0.0.1

Problem solved (Under the conditions I specified). If they’re connected to a service you can of course either restart the service (which will shutdown the connection) or you can drop their socket (via firewall or whatever else).

The one thought (or question) that some may have: why not just add an ip{,6}tables rule that drops their IP to the floor? Well, two reasons. Firstly, its less fun (not to mention they’re still going to be connecting that far.. and sending data). Second, if they’re going to be resolving and connecting, I’d rather them resolve to their own computer and therefore they only communicate with me via DNS and the rest is on their own end.