The gain or loss on the sale of rental property is reported on Form 4797 (PDF), Sales of Business Property. Schedule D (Form 1040) (PDF), Capital Gains and Losses, is often used in conjunction with Form 4797.

For further information, refer to Publication 544, Sales and Other Dispositions of Assets, Publication 550, Investment Income and Expenses, the Instructions for Form 4797 (PDF), Sales of Business Property, and the Instructions for Schedule D (PDF), Capital Gains and Losses.

Rental property is income producing property, and as such, considered business property. The loss on the sale of rental property is reported on Form 4797 (PDF), Sales of Business Property. The ordinary loss is then normally transferred to line 14 of Form 1040 (PDF).

What you have heard about is a transaction called a like-kind exchange.  A like-kind exchange, when properly executed, can postpone the recognition (taxation) of gain essentially by shifting the basis of property sold to like-kind replacement property.

The basis of the property that you acquire in a like-kind exchange is generally the same as the basis of the property that you transferred.  However, if you transfer money or other property (not like-kind) in addition to like-kind property, your basis in the property acquired is the basis of the property given up, increased by the amount of money or other property transferred.  

There are several rules and restrictions that must be strictly adhered to in order for a successful exchange to take place.  For a sale of property and the acquisition of other like-kind property with the proceeds, you must avoid actual or constructive receipt of the proceeds.  You avoid actual or constructive receipt of the proceeds if you comply with one of the safe harbors set forth in the Income Tax Regulations or certain other publications of the Internal Revenue Service.

You may be able to exclude much of the gain from the sale of your main home that you have also used for business or to produce rental income if you meet the ownership and use tests.  However, you may not exclude gain from the sale or exchange of your main home if it is allocable to periods of non-qualified use as detailed in Publication 523, Selling Your Home.

In addition, if you were entitled to take depreciation deductions because you used your home for business purposes or as rental property, you may not exclude the part of your gain equal to any depreciation allowed or allowable as a deduction for periods after May 6, 1997.  If you can show by adequate records or other evidence that the depreciation deduction allowed (did deduct) was less than the amount allowable (could have deducted), you may limit the recognized gain attributable to depreciation to the amount of the depreciation allowed.

How you report the sale of the home depends on its use in the year of the sale.

If the ownership and use tests are met and there is no business or rental use in the year of the sale, then the gain, exclusion, and depreciation-related gain that can’t be excluded should be reported on Schedule D (Form 1040) (PDF), Capital Gains and Losses, and Form 8949 (PDF), Sales and other Dispositions of Capital Assets, as described in Publication 523, Selling Your Home.

If the ownership and use tests are met but there is business or rental use in the year of sale, then the sale of the business or rental part should be reported on the Form 4797, Sales of Business Property.  Form 4797 takes the business or rental part of the gain, the exclusion, and depreciation-related gain that can’t be excluded into account.  

However, if the part of your property used for business or rental use is within your home, you do not need to allocate gain on the sale of the property between the business or rental part of the property and you do not need to report the sale on Form 4797.

Report the exchange of like-kind property on Form 8824 (PDF), Like-Kind Exchanges. The instructions for the form explain how to report the details of the exchange. Even though no gain or loss is recognized, you must report the exchange.

If you received money or other property (not like-kind) as part of the exchange, gain is recognized to the extent of the other property and money received, but a loss is not recognized.  You must report recognized gain on Form 4797 (PDF), Sales of Business Property, and Schedule D (Form 1040) (PDF), Capital Gains and Losses. Refer to Chapter 1, Gain or Loss, in Publication 544, Sales and Other Dispositions of Assets, which has a detailed section on qualifying like-kind exchanges.