Well, you need to keep all the sensors that the PCM/ECU needs to operate the engine properly, or at some point it's not going to meter out the fuel correctly (this could have almost no impact to severe impact). However, if you have the donor vehicle you should be able to drop it in fairly completely. If you break a few sensors, plan a trip to the scrap yard to find the ones you need...
I did an upgrade on a Suzuki Samurai from a 1.3L carb to a 1.6L fuel injection and wow, what a difference -- not just in the increased displacement, but the smoothness of throttle response and no real noticable power loss at altitude. Also, no chance of fuel sloshing and starving the carb on bouncy terrain
The biggest challenge was cleanly splicing in the new ECU with the existing wiring harness... but all the sensors I did retain in their relatively stock location (from a Sidekick).
I have also retrofit a 1979 Jeep J-10 360 V8 with a Holley Pro-Jection aftermarket fuel injection system -- huge difference! Clean, responsive power, actually felt more powerful and more torque (the carb was probably really screwed up) and better mileage. This was even easier to install as there really wasn't much splicing to be done - just add in the new Holley harness and properly tie in power, etc. I did this back in 1997 and drove that truck from Chicago to LA pulling a heavy trailer - ran great.
So, I'm a huge fan of EFI upgrades to non-EFI vehicles.