So, the main point of having the idle speed at a given place is to set the minimum speed at which the motor will run reliably, do at least minimal battery charging, keep oil circulating sufficiently through the motor and (and oil coolers if the bike has one), and (in the case of a liquid cooled motor) sufficiently flow coolant through the radiator. Perhaps equally importantly, you want the motor idling at a point where it will wind smoothly and without hesitation (or possible stalling) when you go to open the throttle.
Which is a very long winded way of saying, "it's generally best to set the idle speed where the manufacturer recommends it".
As for straight pipes, they may sound wicked, but they will definitely affect the fuel mixture--in general, leaning it out through out the throttle range. It may in some cases lead to overheating, and burned exhaust valves. So if you're going to go to a straight pipe, you'd definitely want to look at retuning all the fuel circuits (idle/midrange/WOT) to compensate. You'll also want to move an unrestricted air filter as well (like a K&N) to help match the exhaust adjustment.
For the record, it's quite possible to wind up losing power with a straight pipe relative to where you started out, as even factory pipes are tuned to some degree going back a number of years now. The manufacturers are (in general) always looking for that ideal compromise between muffling and power...