I would like to know how to find a backdoor on my computer?
I have a machine running Mac OS X and I suspect there may be a backdoor installed.
What forensic techniques might I use to determine if any backdoors exist in the system? Additionally, what network configuration options might l want to put in place (firewall or IDS rules, for example) to help limit or detect the exploitation of backdoors in my system?
The efficiency is not a variable here. The trade off is assurance of integrity against resources expended. To achieve complete assurance that your system has perfect integrity (i.e. no one may use your system without your approval) you would need to expend an almost infinite amount of resources.
Your question of how to find a backdoor on my computer can be answered as - you would need an operating system with much stricter partitioning and then OS X. At the extreme you need a specialized processor which provides strict physical separation of data and control (Harvard architecture as opposed to von Neumann architecture). Given the number of system components outside your control (cpu, motherboard, network card, OS, and other software) even an expert would have difficulty achieving a high integrity assurance of a system without outside help. Given that you are not a malware expert the best you can hope for is reduction of risk by reduction of exposure, vulnerability management, and reduction of visibility to threats. Limiting exposure means steps like reducing the total number of hours the system is connected to a network, reducing the size and scope of sensitive data stored on the system, and reducing downloading and installation of software.
Vulnerability management means tracking all the components of your system and continually upgrading or patching any vulnerable components. This is primarily software, but could additionally be a network card or peripheral device. It means monitoring of sources of your OS and applications for alerts about vulnerabilities and reconfiguring or patching your system as needed.
Reduction of visibility to threats means not advertising where or what your system is and what it contains. The easiest way to illustrate this is by showing the opposite. Don’t post on facebook that you started taking credit cards for your small business using your Macbook. This alerts a potential attacker to a valuable target (the credit card numbers) as well as what kind of vulnerabilities it may have.