The answer is definitely YES there is a difference but what that difference is depends on who is answering the question.
Let's start with the definition of SEO or Search Engine Optimisation as there is a clean and consistent definition. According to Wikipedia, Search Engine Optimization is the process of improving the visibility of a web site or a web page in search engines via the “natural" or un-paid ("organic" or “algorithmic") search results. At its most basic level SEO is the process of changing your web site (structure and content) and building links to your site so that it eventually appears on page one of Google in the unpaid or white central area results for particular search terms.
Now let's see what is the difference between SEO and SEM. SEM stands for Search Engine Marketing and even Wikipedia has two definitions. In the broadest sense of the word, search engine marketing is “a form of Internet marketing that seeks to promote websites by increasing their visibility in search engine result pages (SERPs) through the use of search engine optimization (SEO), paid placement, contextual advertising, and paid inclusion. " So search engine optimization is one component of search engine marketing. The second definition again from Wikipedia but referenced in many other online articles, is that “search engine marketing (SEM) target paid listings". It is the use of pay per click ads, banner ads, paid directory listings etc to drive traffic to a site. So if you are confused about the terms, don't worry, so is the internet marketing community too.
The end aim is to drive visitors to a website, keep them there, keep them coming back and get them to do something on the site - buy, sign a petition, participate etc. You should start off with SEO - making sure your site is logically structured, your pages title include the keywords you want to rank for, your description meta tags well written and your headings engaging and also containing the keywords. Once your website is built, you need to submit it to the directories, some of which will be free and some of which are paid. Only select appropriate directories to list in whether they are free or paid. Once that is done, you should use paid advertising to start getting visitors to your site and build your customer database.
Over time switch more funds from paid advertising to search engine optimisation so that your traffic is free rather than paid but there is nothing like paid traffic to get things going. If you have the money, get the professionals to do your keyword research, optimise your website and conduct link building campaigns for you. Outsource your pay per click advertising too so you can focus on growing your business. If you don't have the funds then there are plenty of free DIY internet marketing resources and pay per click training on the internet so don't despair - it is possible to do it yourself, it will just take a little longer that's all.