The Importance of Change and Differences

Ever since micro-organisms set the pace for life on this planet, the earth has been a dynamically changing environment with regard to the life forms which it has supported all along. Changes were necessary – so life evolved from the sea to the land. It would be appropriate to mention the reign of dinosaurs on this planet to justify the impact of change on the environment. It is well known that dinosaurs were the largest and most dominant species during the Cretaceous, Jurassic and Triassic periods. But all of a sudden the entire species fell out of the ecological cycle. Extinction! Among countless theories as to what realized the extinction theory, the best claim is not that of a meteor – as most of us have come to accept. There is a little known theory that the dinosaurs underwent an evolutionary shift which was detrimental to their very existence on this planet. This seems to be the most substantial one in the absence of substantial geological evidence of the terrific impact that such a large meteorite would have caused.

The wonders of the planet have always been in change. In the absence of an evolutionary mechanism, Man as we know it, would never have set foot on this planet. It is this evolutionary mechanism of nature that works behind the scenes when Man chooses to be different from his surroundings. It is the pursuit of all human beings to have a recognizable identity in his immediate environment. It is this desire to be special that drives us all to be different in our own ways.

Man is not equal! Changes are necessary and inevitable. It is in learning to accept our differences as human beings that we can assert our claim as the most intelligent species on this planet today.

The Purpose of Existence and The Convergence

It has been really the pursuit of man through all the ages to identify himself as a “being” in this world. However, there is a still a big “WHY” as to the reason behind our existence in this world. This reason might be far removed from the “HOW” of our existence which is of course fundamentally explained by evolution theories.

We keep identifying ourselves as human beings and annotating our identity in this world as the most intelligent species that ever breathed life on this planet. We love to identify ourselves as a being in this world. We like to think of ourselves as separate entities – each with their own separate and independent functionalities

However, we are not alone in this world. I would hesitate to identify myself, or anyone else for that matter, as an independent entity in this world. The issue here is really what we call symbiosis (co-existence for the layman). Putting it simply, we might seem to be co-existing individuals. On a more cellular level, or even diving down to the atomic level, we are composed of the very atoms that make up other entities in this universe. As such, considering the human being to be a distinct entity would be a gross misappropriation of the reality at hand. The reality is that we are essentially part of a large and in-comprehensible distributed system.

Note: Grossly speaking, a distributed system is a collection of atomic and simple elements that perform complicated tasks through individual contribution and interactions. A classic example in this regard would be any of the animal colonies that we observe everyday. Whether it is a flock of birds performing an intricate pattern in the skies, or a colony of ants marching to and fro from their nests to food sources; distributed systems are commonplace. Each entity is quite simple as far as their individual functions are concerned. However, the interactions result in a more complex behavior that what can ever be expected out of one such solitary entity. The individual entities have little realization about the complexity that the task involves and even little about the nature of the complex phenomenon that they are a part of; thus giving rise to collective intelligence.

What I want to drive home from this allegory is that we are not individual entities in any respect. What we are is part of one large complex and distributed system, which I might refer to as the universe. What lies beyond the universe is also part of the same unified complex system that we are all a part of. However, I would limit my definition of the complex system to be the universe as we know it. As I have already mentioned, an individual entity has little or no idea about the complex system that they are contributing to, it is highly improbable that even with all our knowledge, we would ever be able to grasp the complexities that lie in the system that we all contribute our part to.

As another example, I can safely assume that an individual atom inside our body is, with all probability, not even aware of the fact that it is contributing to all the delicate and coordinating functions that make a living being stay alive. So it becomes increasingly difficult for entities at lower levels to grasp the true complex nature of the more top level functionalities in a distributed system.

We can never justify our existence in this world by simply identifying the elements and their functions that drive our body. For it is essentially an ever-debatable question as to what comprises our body. If we consider that whatever helps us in our life functions, such as the cells in our body as a part of our being, we are missing out on the essence of it all. On a broader perspective, if we consider whatever helps us exist to be a part of us, we must consider everything around us, our society, our environment, and so on, till we finally realize that it is the universe that comprises what we physically are. Logically this comes down to saying that “my existence is the universe”. Now, if our physical existence spans the universe, can our minds be far behind?

What would keep on driving us is therefore the need to explore the universe to keep on learning more about the way it works. That is the way we must tread to uncover the true objective of the universe, which would essentially be the answer to the WHY of our existence.

The why of anything defines purpose – purpose of existence. So, it is imperative that we understand the purpose of our existence and to know why we exist – awareness, consciousness, and whatever else comes along the way!

Regarding consciousness

I presume consciousness is still a vague term which has hardly found a meaningful explanation as of yet. After all, we are not even aware of the fact that whatever we perceive or feel is even remotely related to reality. For the present discussion, I agree and assume that consciousness is the mechanism that makes an entity self-aware amongst other criteria as well.

Sometimes, we may tend toward consciousness - but the knowledge of achieving consciousness is elusive. Even if we reach consciousness - can we be conscious enough to realize our consciousness with our consciousness? It is impossible to fully understand the mechanisms that drive the human brain and our emotions and thought processes along with it.

Whatever is thought or known is the result of numerous complicated neural interactions within the substance that lies enclosed within our cranium. If a part of the cranium tries to comprehend the mechanisms and equations that govern the entire brain, it would be analogous to trying your best at pushing a car forward while sitting in the car itself!

Musings on the purpose of life

The differences and the convergence are thinly balanced factors – all contributing in unison to the gradual development of human minds and society. As such, the entities that comprise the system that drives us are all different in their own respect; although we find similarity in the very universe that we dwell in.

The objective of the existence of human beings is nothing special, yet something as commonly awe-inspiring as it can get. It is to aid in the complex system functions of the superset of all subsistence – the universe.

Human beings are part of whatever we can see around us. However, it is also painfully true and sadly enough, food, shelter and reproduction are still the three most basic instincts (in order perhaps) that human beings respond to. When we get time off from the activities that satisfy our necessities, we try to conjure up images of being a far more superior species than those around us. This might be true to a certain extent, but that's solely due to the complexity of our thought processes which is perhaps enhanced by the size of our brain when compared against other species.

This is something purely anatomical and as such, there is no other reason why we should consider ourselves to be a far more superior species with a purpose different from those of others.

The purpose of the human species is like any other organism today and others before our time..... The purpose is survival.

It is the basic instinct of all life and the purpose of our existence is to ensure that it holds true for our species in our time and for generations to come.

However, this interpretation is only part of the truth! If the reader thinks that that’s all there is to it, try and imagine that you are part of a large and complex organization that everyone respects; imagine the dignity that you would take out of it. Now replace the imaginary organization with something much larger by exponential proportions. Replace it with the universe. And watch the pride and self-respect for your part in it jump over the top of your hesitant feelings.

The difference lies in how we are remembered when our physical existence has parted from life. It’s not survival that defines our consciousness; it’s the way we choose to tread the path to survival that defines who we are and how we shall be remembered. This gave birth to consciousness (self-awareness if you may call it) and sustains the spirit that ushers in the urge for exploration, discovery and everything else that we have achieved so far.

I pay homage to the solitude as well as the company that has brought in such fears and hopes into my musings and the imaginations they conjure are what inspired the following compositions; in change, differences, purpose, symbiosis and consciousness.

The I in AI

We stand today at the verge of a major crossroad in the history of mankind and computers. Until today, it was assumed that computers were what we want it to be, it will obey what we teach it to do, it will carry out the will of its master as would a pedestrian slave. But times are changing and so are my thoughts. We are not far away from a future where a modern day programmer could have the role of an unprivileged, unemployed person. What if we could make a program which could generate other programs according to its will? A machine self sustaining, self repairing, self optimizing – a machine which could chart out a solution for every possible problem in the same methodical and analytical way that we, humans are used to doing – a machine which could think.

For starters let us assume an ordinary day in the life of a human being, say some fifty years from now. Morning begins with an alarm, but with a difference. The alarm understands, it sympathizes with you. It realizes that it is cold outside, and you could very well do with an extended sleep. After a while, a fleshy touch gently shakes your shoulder and whispers in your ears that it is time for you to get up. You wake up to find the cup of coffee already placed at your bedside.

Work is no problem, it is a simple matter of feeding the requirements of the job to the machine and the machine devises a logical set of solutions to the problem. When you interrupt and say that a certain task for which the solution is done could have been interpreted in a better manner, it logically reasons and works with you to find the best possible alternative. It admits when it is wrong and says that it is human to make mistakes. You get startled at this statement, but then realize, it is human after all. It learns, rationalizes, infers and predicts. The machine of the future churned out from the assembly line is going to be a child who has been taught the raw details of interaction and science. But the pathways embedded in birth are not absolute; it alters with training and experience. Yes, I am talking about Artificial Intelligence in its absolute form – a humanoid.

When we face a critical problem in the machine of today, the only possible solution is to manually decipher the cause of the problem and eradicate it. Wouldn’t it have been better had the machine itself dealt with its internal and runtime anomalies and self adjusted in a repair phase keeping transparent to the operator, the truth of its state. This would, in effect produce a self sustaining machine which I was talking of earlier.

All this is like a dream as we still reside in the infancy of Artificial Intelligence. But with breakthroughs in Natural Language Processing and spirited research in distributed computing, we are slowly but surely moving forward in realizing our dream. This is not unlike perpetual motion though, where man had to face several obstacles before realizing that it was not possible. AI is the beginning of the future of science and human interactions with their mechanical counterparts. The concept of such an absolute machine as I mentioned is the manifestation of all the good things AI has to offer, and it will be just the beginning of a brave new world.

Chaos and Uncertainty

Unreal optimism regarding certain so-called obvious and explainable phenomenon has been the hallmark of all of humankind’s scientific history. However, after countless counter-intuitive theories and deliberations, it is high time when we speculate on what is possible and what is not. Take Von Neumann, for example, who built his first computer with an intention of predicting and controlling the weather, among other intentions. However, we know today, that we cannot forecast the weather. It is true, however, that we might make probabilistic predictions for one or two days ahead, without making significant errors, but no meteorological office can dare predict even a week ahead. Beyond two to three days, the world’s best forecasts are speculative, and beyond six or seven, they are worthless. The solution to this problem does not concern building faster and more efficient computers. For we are dealing with systems that are complex, non-linear in nature – we are dealing with systems which have a sensitive dependence on initial conditions – we are dealing with chaos.

Chaos is a phenomenon seen in our everyday lives. Whether it is the turbulent flow of water from a faucet, or the pillars of smoke that rise from chimneys in industrial towns, or a boiling kettle, chaos is everywhere. Classical methods can never calculate where a piece of paper might end up when a certain period has passed after it has been thrown over a waterfall, can it?

Butterfly effect is a term coined specifically indicating the futility of weather forecast attempts. In the 1950’s and 60’s, the philosophical heart of science was stated somehow like this – ‘Given an approximate knowledge of a system’s initial conditions, and an approximate understanding of natural law, one can calculate the approximate behavior of the system’. This would be true for linear systems where a small error in fixing the position of the Halley’s Comet would lead to a tiny error in the prediction of the next appearance. However, in non-linear systems, approximation of the initial conditions (like ignoring the effect of a butterfly fluttering its wings) leads to rapid divergence from the desired path and could lead to a tornado in South America, even if clear weather had been predicted for that day. Since with current technology, we can never measure the state of the universe at a snapshot instant, and even if we had the technology, the measurements would not be correct, since by Heisenberg, we change whatever we observe, we have to declare the future of weather forecasting as ‘doomed’.

Addressing the same issue somewhat differently, can physics approximately predict the final position of a billiard ball after it has been struck across the table? The answer is yes, for a few collisions, the equations governing motion and friction can be useful. But let us find out whether such a prediction is possible for a large number of collisions. For starters, let us assume we have a very powerful computer and sensors located all over the earth. All the initial data is correctly recorded. However, we ignore the effect caused by a falling leaf in some distant planet some light years away. Let us assume that the ball deviates by 10^(-140) degrees due to the leaf. For linear collisions, this error would lead to a minor deviation from where we predicted the ball to end up at. But when a billiard ball strikes another, the collision is non-linear. Let us consider that the deviation from prediction is amplified 10 times for each non-linear collision. So, after one collision, the error is 10^(-139) degrees, after two, it is 10^(-138) degrees, and so on. After the 140th collision, it is 1 degree, after the 141st, it is 10 degree, until at the 143rd collision the error in prediction has amplified to 1000 degrees. It is like a fractal where minor changes can lead to a radically different patterning altogether. We even do not know in which direction the ball is moving, let alone, predict the final approximate position of the ball.

Chaos exploits the disorder in such systems and finds out the underlying order in the apparently disorderly systems. Had chaos merely pointed out the disorder, it would have been of little use as a science. Scientists believe that Chaos holds the key to the understanding of the universe. Population biologists, mathematicians, physicists, all recognize chaos as a potent tool for the understanding of complex systems – systems whose behavior cannot be predicted classically.

I hope that this discussion has led to the reader being in a position to appreciate the importance that chaos plays in our everyday lives. I have purposefully refrained from discussing what chaos theory is all about and concentrated only on the background information. The interested reader might find it very “fulfilling”, if I may say so, to go through a good book on the theory of chaos to find out more on what this science has to offer. Happy hunting readers!

Current Spam Techniques and Trends

Everytime I look at the spam folder of my e-mail account, I observe that existing spamming techniques have evolved rapidly - faster than I would have liked!

I would like to express my observations regarding the current scenario of e-mail spam.

Previously (a couple of years back) we had spam e-mail as a regular visitor to our e-mail accounts. However, detecting these mails were a lot easier since they used to be composed of almost the same set of keywords that were present in almost every other junk mail. Our old junk mail filter used to check incoming mails for the frequency of all words in it. It would then draw a comparison between the frequencies of the words in the incoming mail against the frequencies of the words in standard spam mail. So the spam filters were constructed using a Bayesian probabilistic mechanism in which the incoming e-mail was classified as spam if the frequency of certain words in it followed the frequency of the same words in standard spam mail. There were certain other variations to this technique; however the practiced mode still remained probabilistic measures and word frequency analysis.

Coming back to the point I was trying to place, I opened my spam folder in my e-mail account to check for some false positives (genuine mail or "ham" that has been incorrectly classified as "spam") that I was expecting. The following are my observations from the nature of mail in the spam folder:

Standard spam e-mails come from legitimate looking addresses and he sender names are not some cryptic business or company names - they are names that are commonly used. (E.g. Mary Sanders): The sender e-mail addresses are often genuine e-mail addresses that have been hacked or infested with a worm that has resulted in compromising the account (converted it to a spam sender). A couple of years back; we could have easily identified spam by looking at the sender name or the sender e-mail address. However, in recent times, spam is being transformed into a form that represents a genuine e-mail. Indeed it has become difficult to distinguish between a standard spam and a ham today even by visual inspection. Imagine the difficulties faced by an automated tool that is supposed to classify the messages coming in daily into your inbox as spam or ham.

These mails have a friendly subject line (often prefixed by a Re: to indicate that the e-mail was a reply from one of your sent mails): I was surprised to see that some of the mails prefixed by the subject line Re: contain reply attachments which indicate that I sent a mail to that address and the junk mail is simply a reply to what I sent. I am yet to comprehend whether my account has been compromised as in the previous observation and the Re: prefixed e-mails I receive are simple challenge responses sent by the receiving e-mail address server. Alternatively, this might be another twist in the tale from the spammers where they construct the spam mail by disguising it as a reply to a genuine e-mail.

The content of these e-mails generally contain strings of literary works (mostly quoted from popular literary works such as Macbeth, Hamlet, etc): Spammers have now started sending fragments of well known literary works along with the e-mails in order to damage the frequency database that standard anti-spam utilities use. If we are using one of the dated Bayesian anti-spam utilities and we start classifying modern e-mail as spam, then it is very likely that the spam-word-frequency database will be unnecessarily adulterated (Bayesian poisoning) with the wrong frequencies and result in more false positives than warranted. The solution lies in updating your anti-spam filter with one which uses a more of a heuristic approach rather than a simple probabilistic estimate.

It is possible for the spammer to disguise certain words that the spam filter recognizes as spam mail text: For example, it is common for a spam filter to filter out mail containing the word viagra, for example. Unfortunately, it has become common for spammers to disguise these words with intermediate special characters so that they do not appear as the words under the scanner (for example, the word viagr@ can be used instead). With html mails becoming commonplace, invalid html tags are also used in between these words for the purpose of throwing the spam filter off-track. Ironically, since the html interpreter ignores invalid tags, the text appears to the recipient as a normal message which serves the spammer's purpose.

Some spam mails contain an image attachment that contains the advertisement: Image attachments sent along with spam mails are very difficult to detect and classify as spam. Spammers have resorted to sending their advertisements as images that contain the ad. This is indeed difficult for an automated system to detect. Even if we used an optical character recognition utility, the process would not be successful as I have noted that the characters in the images are often scrawled or blurred and the background littered with misleading pixels (as in standard Image verification text that appear on web-forms to prevent automated form submissions). These result in the image being almost indecipherable to the OCR utility, but at the same time the characters remain visually discernable to the person viewing it.

It is not always a requirement that a person has to fill up his e-mail address in a web-form in order to receive spam mail: We might receive spam even if we use an account that has not been given away in a web-form that compromises your mail account to spammers. Nowadays spammers have also resorted to generating lists of arbitrary e-mail addresses that contain commonly used names or phrases. These addresses become the targets of spam mail. The solution would lie in judicious use of the special characters in your address that the account provider permits. For example, it might be wise to use Sanjit_11_b@xyz.com rather than using Sanjit@xyz.com. But the drawback of this technique might be that people will tend to forget your e-mail addresses more often.

All said and done, spam is becoming a major problem for us all. I usually get around 20 messages a day in my spam folder (people might get less if they are luckier). Some false negatives (spam which is not detected) also creep in sometimes in my inbox and a few false positives into my spam folder as well. Cyber laws will hardly be a deterrent to this nuisance. And as we all know cyber laws mostly reside in cyberspace where they are known to exist but hardly ever exercised, leaving it all to the authorities would result in getting stuck with this nuisance for God knows how long. Getting a proper solution to this problem would perhaps be something that we’ll forever dream of as spamming techniques are clearly outrunning the anti-spam filter technologies. We could go on and change the existing e-mail system and come up with a new one which enforces security, digital signatures and so on, but even then newer techniques to counter the security measures are sure to crop up and leave us baffled.

For now, the best we can do is hope for something to develop while we keep on clicking the “Report as Spam” & “Delete” buttons for the spam that we receive.