1/ According to the Baltzan, Business Driven Information Systems 2ed.
Electronic data interchange (EDI) is a standard format for exchanging business data. And EDI is becoming increasingly important as an easy mechanism for companies to buy, sell, and trade information
2/ According to Answers.com
(Electronic Data Interchange) The electronic communication of business transactions, such as orders, confirmations and invoices, between organizations. Third parties provide EDI services that enable organizations with different equipment to connect. Although interactive access may be a part of it, EDI implies direct computer-to-computer transactions into vendors' databases and ordering systems.The Internet gave EDI quite a boost. However, rather than using privately owned networks and the traditional EDI data formats (X12, EDIFACT and TRADACOMS), many business transactions are formatted in XML and transported over the Internet using the HTTP Web protocol
3/ According to BusinessDictionary.com
The predecessor to electronic commerce, EDI has been suitable only for large North American and European corporations and banks. Firms using EDI are interconnected through a global computerindependent of internet although attempts are underway to integrate the two networks. It facilitates computer-to-computer exchange of electronic documents (such as purchase orders, advance shipment notices, and invoices) without human intervention or human readable (paper or electronic) documents. EDI eliminates manual re-keying of data, cuts order processing costs, increases data accuracy, improves cycle time, and makes just-in-time deliveries possible. Like internet it is a standards based system independent of the type of computer hardware and software employed.
Electronic data interchange (EDI) appears in the definitions of the following terms: Electronic Data Interchange for Administration, Commerce, and Transportation (EDIFACT), electronic business (E-Business) and continuous acquisition and life-cycle support (CALS)
More definition at Wikipedia.org
Electronic Data Interchange (EDI) refers to the structured transmission of data between organizations by electronic means. It is used to transfer electronic documents from one computer system to another (ie) from one trading partner to another trading partner. It is more than mere E-mail; for instance, organizations might replace bills of lading and even checks with appropriate EDI messages. It also refers specifically to a family of standards, including the X12 series. However, EDI also exhibits its pre-Internet roots, and the standards tend to focus on ASCII (American Standard Code for Information Interchange)-formatted single messages rather than the whole sequence of conditions and exchanges that make up an inter-organization business process.
In 1992, a survey of Canadian businesses found at least 140 that had adopted some form of EDI, but that many (in the sample) "[had] not benefited from implementing EDI, and that they [had] in fact been disadvantaged by it." 
The National Institute of Standards and Technology in a 1996 publication  defines Electronic Data Interchange as "the computer-to-computer interchange of strictly formatted messages that represent documents other than monetary instruments. EDI implies a sequence of messages between two parties, either of whom may serve as originator or recipient. The formatted data representing the documents may be transmitted from originator to recipient via telecommunications or physically transported on electronic storage media.". It goes on further to say that "In EDI, the usual processing of received messages is by computer only. Human intervention in the processing of a received message is typically intended only for error conditions, for quality review, and for special situations. For example, the transmission of binary or textual data is not EDI as defined here unless the data are treated as one or more data elements of an EDI message and are not normally intended for human interpretation as part of online data processing." EDI can be formally defined as 'The transfer of structured data, by agreed message standards, from one computer system to another without human intervention'. Most other definitions used are variations on this theme. Even in this era of technologies such as XML web services, the Internet and the World Wide Web, EDI is still the data format used by the vast majority of electronic commerce transactions in the world
EDI and other similar technologies save a company money by providing an alternative to, or replacing information flows that require a great deal of human interaction and materials such as paper documents, meetings, faxes, etc. Even when paper documents are maintained in parallel with EDI exchange, e.g. printed shipping manifests, electronic exchange and the use of data from that exchange reduces the handling costs of sorting, distributing, organizing, and searching paper documents. EDI and similar technologies allow a company to take advantage of the benefits of storing and manipulating data electronically without the cost of manual entry. Another advantage of EDI is reduced errors, such as shipping and billing errors, because EDI eliminates the need to rekey documents on the destination side. One very important advantage of EDI over paper documents is the speed in which the trading partner receives and incorporates the information into their system thus greatly reducing cycle times. For this reason, EDI can be an important component of just-in-time production systems.
According to the 2008 Aberdeen report "A Comparison of Supplier Enablement around the Word", only 34% of purchase orders are transmitted electronically in North America. In EMEA, 36% of orders are transmitted electronically and in APAC, 41% of orders are transmitted electronically. They also report that the average paper requisition to order costs a company $37.45 in North America, $42.90 in EMEA and $23.90 in APAC. With an EDI requisition to order costs are reduced to $23.83 in North America, $34.05 in EMEA and 14.78 in APAC.
The United States health care system consists of thousands of different companies and other entities. In 1996, the Health Insurance Portability and Accountability Act (HIPAA) was enacted. In short, it set down standard transaction sets for specific EDI transactions and mandated electronic support for every insurance company in the United States for these transactions. While the benefits of EDI are numerous and only increase with increased volume, the drawbacks, though not directly related to EDI itself, include managerial problems in the support, maintenance and implementation of EDI transactions.
This all comes at a massive cost in time and management as a company may attempt to support a broad range of transactions with a broad range of entities. This example is an extension of the lack of strict standards across implementations, transactions and methods.