no, not because of the memory it takes or CPU cycles burned (does not really matter when you have 4 GB notebook with Core2Duo).
Few days ago, a good friend from old country (well, technically not anymore as he also moved within the EU) made me aware of this presentation "Silver Needle in the Skype" (link points to fairly large PDF file) by Philippe Biondi, & Fabrice Desclaux.
To fully digest and fully comprehend the content requires way more time than I am willing to invest - and to make meaningful arguments for or against conclusions does require much deeper special knowledge. It is interesting view into the deep internals of how Skype works and also provides very interesting references to tools available for this kind of exploration. I am not going to stop using Skype just because there is a chance that Skype could possibly be a backdoor or something not so innocent. There can be after all perfectly honest reason for all the obfuscation and anti-disassembling measures - to protect the IP against competition. Or it can be in order to hide something else ? We will probably never know.
But I am not letting Skype start as the machine boots anymore and shut it down after I am done with my call. In other words, you will not find me online on Skype very often :-).