It is really a crap shot for self employed individuals not paying taxes and it depends. In some cases you can go ten or more years, in other cases they'll catch you the next year. It really depends.
Computer systems are CRA are always looking for unusual activity. Remember that your bank forwards interest statements to them. A program could easily trigger by noting that you have earned interest but didn't file that year. (That isn't that unusual as perhaps you just didn't own any taxes). If you are getting provincial health insurance computers may note that you have declared your income and yet didn't file taxes. It could be random audits of your employer -- since they will be declaring your work as an expense. If you do file income taxes and just fail to declare that income, it becomes much harder for them to notice (other than auditing you), but you now risk extremely serious offenses for filing a false income tax return.
If you are caught, in most cases they will just make you pay plus the fines and interest. They could file charges and jail you... but the reality is that tax collectors are paid on commission and they (and their bosses) would rather be paid than waste court time. Still it can be nerve-racking and you typically need legal (or accounting) representation to deal with CRA.
There are also a number of non-CRA issues by not filing. You won't get credit for CPP or EI, so you can't collect these. You won't have a notice of assessment which means that you can't get loans, credit, lines of credit, life insurance, etc. It makes obtaining provincial health insurance risky/difficult, since they verify income with income tax returns. It also makes it more and more difficult to file.
I would recommend talking with a tax preparation office. Have them run your 2011 taxes and see what you owe. If you didn't earn much... you may not owe any taxes (other than CPP and EI) or it might not be so bad. Worst case, you can file and make a token payment and then start paying regularly. They rarely come after you if you are paying.