WordPress caching is the fastest way to improve performance. If your site is getting hit right now install W3 Total Cache or WP Super Cache.
Plugins like W3 Total Cache or WP Super Cache can be easily installed and will cache your WordPress posts and pages as static files. These static files are then served to users, reducing the processing load on the server. This can improve performance several hundred times over for fairly static pages.
If your posts/pages have a lot of dynamic content configuring caching can be more complex. Search for "WordPress cache plugin" for more info.
Look into HTTP Cache-Control (specifically max-age) and Expires headers, as well as Entity Tags for more information.
Web server caching is more complex but is used in very high traffic sites. A wide range of options are available, beyond the scope of this article. The simplest solutions start with the server caching locally while more complex and involved systems may use multiple caching servers (also known as reverse proxy servers) "in front" of web servers where the WordPress application is actually running. Adding an opcode cache like Alternative PHP Cache (APC) to your server will improve PHP's performance by many times.
For more on this type of server setup check out High Scalability site among others.