In order to get married in Bolivia, you have to enter the country with an "uso determinado" visa (pre-determined use visa). You can NOT get legally married in Bolivia if you come in on a tourist visa. To get the special visa, you will need to go through the embassy in the country you are coming from. It is not guaranteed to be a quick and easy visa to get, so you will want to plan a bit in advance. If you are marrying a Bolivian you will need this visa, if you are both foreigners, you will both need this visa. Legal residents MAY be able to get out of this.
In order to get the special visa, you will need to have (from your home country) a notarized letter from a non-family friend stating that they swear you are single. The notarized letter will need to be translated and the translation will need to be notarized and certified.
The formalities in-country aren't bad, pretty much any notary public can do the paperwork.