Reverse engineering, also called back engineering, is the process by which a man-made object is deconstructed to reveal its designs, architecture, or to extract knowledge from the object; similar to scientific research, the only difference being that scientific research is about a natural phenomenon.:3
There are many reasons for performing reverse engineering in various fields. Reverse engineering has its origins in the analysis of hardware for commercial or military advantage.:13 However, the reverse engineering process in itself is not concerned with creating a copy or changing the artifact in some way; it is only an analysis in order to deduce design features from products with little or no additional knowledge about the procedures involved in their original production.:15 In some cases, the goal of the reverse engineering process can simply be a redocumentation of legacy systems.:15 Even when the product reverse engineered is that of a competitor, the goal may not be to copy them, but to perform competitor analysis. Reverse engineering may also be used to create interoperable products and despite some narrowly tailored United States and European Union legislation, the legality of using specific reverse engineering techniques for this purpose has been hotly contested in courts worldwide for more than two decades.
Software reverse engineering can help to improve the understanding of the underlying source code for the maintenance and improvement of the software, relevant information can be extracted in order to make a decision for software development and graphical representations of the code can provide alternate views regarding the source code, which can help to detect and fix a software bug or vulnerability. Frequently, as some software develops, its design information and improvements are often lost over time, but this lost information can usually be recovered with reverse engineering. This process can also help to cut down the time required to understand the source code, reducing the overall cost of the software development. Reverse engineering can also help to detect and eliminate a malicious code written to the software with better code detectors. Reversing a source code can be used to find alternate uses of the source code, such as to detect unauthorized replication of the source code where it wasn't intended to be used, or to reveal how a competitors product was built. This process is commonly used for "cracking" software and media to remove their copy protection,:7 or to create a (possibly improved) copy or even a knockoff, which is usually the goal of a competitor or a hacker.:8 Malware developers often use reverse engineering techniques to find vulnerabilities in an operating system (OS), in order build a computer virus that can exploit the system vulnerabilities.:5 Reverse engineering is also being used in cryptanalysis in order to find vulnerabilities in substitution cipher, symmetric-key algorithm or public-key cryptography.:6
As computer-aided design (CAD) has become more popular, reverse engineering has become a viable method to create a 3D virtual model of an existing physical part for use in 3D CAD, CAM, CAE or other software. The reverse-engineering process involves measuring an object and then reconstructing it as a 3D model. The physical object can be measured using 3D scanning technologies like CMMs, laser scanners, structured light digitizers, or Industrial CT Scanning (computed tomography). The measured data alone, usually represented as a point cloud, lacks topological information and design intent. The former may be recovered by converting the point cloud to a triangular-faced mesh. Reverse engineering aims to go beyond producing such a mesh, and to recover the design intent in terms of simple analytical surfaces where appropriate (planes, cylinders, etc), as well as possibly NURBS surfaces, to produce a boundary-representation CAD model. Recovery of such a model allows a design to be modified to meet new requirements, a manufacturing plan to be generated, etc.
Hybrid Modelling is commonly used term when NURBS and Parametric modelling are implemented together. Using a combination of geometric and freeform surfaces can provide a powerful method of 3D modelling. Areas of freeform data can be combined with exact geometric surfaces to create a hybrid model. A typical example of this would be the reverse engineering of a cylinder head, which includes freeform cast features, such as water jackets and high tolerance machined areas.
Reverse engineering is also used by businesses to bring existing physical geometry into digital product development environments, to make a digital 3D record of their own products, or to assess competitors' products. It is used to analyse, for instance, how a product works, what it does, and what components it consists of, estimate costs, and identify potential patent infringement, etc.
Value engineering is a related activity also used by businesses. It involves de-constructing and analysing products, but the objective is to find opportunities for cost cutting.
In 1990, Institute of Electrical and Electronics Engineers (IEEE) defined reverse engineering as "the process of analyzing a subject system to identify the system's components and their interrelationships and to create representations of the system in another form or at a higher level of abstraction", where the "subject system" is the end product of software development. Reverse engineering is a process of examination only: the software system under consideration is not modified (which would make it re-engineering or restructuring). Reverse engineering can be performed from any stage of the product cycle, not necessarily from the functional end product.
There are two components in reverse engineering: redocumentation and design recovery. Redocumentation is the creation of new representation of the computer code so that it is easier to understand. Meanwhile, design recovery is the using of deduction or reasoning from general knowledge or personal experience of the product in order to fully understand the product functionality. It can also be seen as "going backwards through the development cycle". In this model, the output of the implementation phase (in source code form) is reverse-engineered back to the analysis phase, in an inversion of the traditional waterfall model. Another term for this technique is program comprehension. Working Conference on Reverse Engineering (WCRE) has been held yearly to explore and expand the techniques of reverse engineering. Computer-aided software engineering (CASE) and automated code generation have contributed greatly in the field of reverse engineering.
Software anti-tamper technology like obfuscation is used to deter both reverse engineering and re-engineering of proprietary software and software-powered systems. In practice, two main types of reverse engineering emerge. In the first case, source code is already available for the software, but higher-level aspects of the program, perhaps poorly documented or documented but no longer valid, are discovered. In the second case, there is no source code available for the software, and any efforts towards discovering one possible source code for the software are regarded as reverse engineering. This second usage of the term is the one most people are familiar with. Reverse engineering of software can make use of the clean room design technique to avoid copyright infringement.
On a related note, black box testing in software engineering has a lot in common with reverse engineering. The tester usually has the API, but their goals are to find bugs and undocumented features by bashing the product from outside.
Other purposes of reverse engineering include security auditing, removal of copy protection ("cracking"), circumvention of access restrictions often present in consumer electronics, customization of embedded systems (such as engine management systems), in-house repairs or retrofits, enabling of additional features on low-cost "crippled" hardware (such as some graphics card chip-sets), or even mere satisfaction of curiosity.
Binary reverse engineering is performed if source code for a software is unavailable. This process is sometimes termed Reverse Code Engineering, or RCE. As an example, decompilation of binaries for the Java platform can be accomplished using Jad. One famous case of reverse engineering was the first non-IBM implementation of the PC BIOS which launched the historic IBM PC compatible industry that has been the overwhelmingly dominant computer hardware platform for many years. Reverse engineering of software is protected in the U.S. by the fair use exception in copyright law. The Samba software, which allows systems that are not running Microsoft Windows systems to share files with systems that are, is a classic example of software reverse engineering, since the Samba project had to reverse-engineer unpublished information about how Windows file sharing worked, so that non-Windows computers could emulate it. The Wine project does the same thing for the Windows API, and OpenOffice.org is one party doing this for the Microsoft Office file formats. The ReactOS project is even more ambitious in its goals, as it strives to provide binary (ABI and API) compatibility with the current Windows OSes of the NT branch, allowing software and drivers written for Windows to run on a clean-room reverse-engineered Free Software (GPL) counterpart. WindowsSCOPE allows for reverse-engineering the full contents of a Windows system's live memory including a binary-level, graphical reverse engineering of all running processes.
Another classic, if not well-known example is that in 1987 Bell Laboratories reverse-engineered the Mac OS System 4.1, originally running on the Apple Macintosh SE, so they could run it on RISC machines of their own.
Reverse engineering of software can be accomplished by various methods. The three main groups of software reverse engineering are
Software classification is the process of identifying similarities between different software binaries (for example, two different versions of the same binary) used to detect code relations between software samples. This task was traditionally done manually for several reasons (such as patch analysis for vulnerability detection and copyright infringement) but nowadays can be done somewhat automatically for large numbers of samples.
This method is being used mostly for long and thorough reverse engineering tasks (complete analysis of a complex algorithm or big piece of software). In general, statistical classification is considered to be a hard problem and this is also true for software classification, therefore there aren't many solutions/tools that handle this task well.
Although UML is one approach to providing "reverse engineering" more recent advances in international standards activities have resulted in the development of the Knowledge Discovery Metamodel (KDM). This standard delivers an ontology for the intermediate (or abstracted) representation of programming language constructs and their interrelationships. An Object Management Group standard (on its way to becoming an ISO standard as well), KDM has started to take hold in industry with the development of tools and analysis environments which can deliver the extraction and analysis of source, binary, and byte code. For source code analysis, KDM's granular standards' architecture enables the extraction of software system flows (data, control, & call maps), architectures, and business layer knowledge (rules, terms, process). The standard enables the use of a common data format (XMI) enabling the correlation of the various layers of system knowledge for either detailed analysis (e.g. root cause, impact) or derived analysis (e.g. business process extraction). Although efforts to represent language constructs can be never-ending given the number of languages, the continuous evolution of software languages and the development of new languages, the standard does allow for the use of extensions to support the broad language set as well as evolution. KDM is compatible with UML, BPMN, RDF and other standards enabling migration into other environments and thus leverage system knowledge for efforts such as software system transformation and enterprise business layer analysis.
Protocols are sets of rules that describe message formats and how messages are exchanged (i.e., the protocol state-machine). Accordingly, the problem of protocol reverse-engineering can be partitioned into two subproblems; message format and state-machine reverse-engineering.
The message formats have traditionally been reverse-engineered through a tedious manual process, which involved analysis of how protocol implementations process messages, but recent research proposed a number of automatic solutions. Typically, these automatic approaches either group observed messages into clusters using various clustering analyses, or emulate the protocol implementation tracing the message processing.
There has been less work on reverse-engineering of state-machines of protocols. In general, the protocol state-machines can be learned either through a process of offline learning, which passively observes communication and attempts to build the most general state-machine accepting all observed sequences of messages, and online learning, which allows interactive generation of probing sequences of messages and listening to responses to those probing sequences. In general, offline learning of small state-machines is known to be NP-complete, while online learning can be done in polynomial time. An automatic offline approach has been demonstrated by Comparetti et al. and an online approach by Cho et al.
Other components of typical protocols, like encryption and hash functions, can be reverse-engineered automatically as well. Typically, the automatic approaches trace the execution of protocol implementations and try to detect buffers in memory holding unencrypted packets.
Reverse engineering is an invasive and destructive form of analyzing a smart card. The attacker grinds away layer after layer of the smart card and takes pictures with an electron microscope. With this technique, it is possible to reveal the complete hardware and software part of the smart card. The major problem for the attacker is to bring everything into the right order to find out how everything works. The makers of the card try to hide keys and operations by mixing up memory positions, for example, bus scrambling. In some cases, it is even possible to attach a probe to measure voltages while the smart card is still operational. The makers of the card employ sensors to detect and prevent this attack. This attack is not very common because it requires a large investment in effort and special equipment that is generally only available to large chip manufacturers. Furthermore, the payoff from this attack is low since other security techniques are often employed such as shadow accounts. It is uncertain at this time whether attacks against CHIP/PIN cards to replicate encryption data and consequentially crack PINS would provide a cost effective attack on multifactor authentication.
Reverse engineering is often used by people in order to copy other nations' technologies, devices, or information that have been obtained by regular troops in the fields or by intelligence operations. It was often used during the Second World War and the Cold War. Well-known examples from WWII and later include:
Reverse engineering applies primarily to gaining understanding of a process or artifact, where the manner of its construction, use, or internal processes is not made clear by its creator.
Patented items do not of themselves have to be reverse-engineered to be studied, since the essence of a patent is that the inventor provides detailed public disclosure themselves, and in return receives legal protection of the invention involved. However, an item produced under one or more patents could also include other technology that is not patented and not disclosed. Indeed, one common motivation of reverse engineering is to determine whether a competitor's product contains patent infringements or copyright infringements.
Reverse engineering of computer software in the US often falls under both contract law as a breach of contract as well as any other relevant laws. This is because most EULAs (end user license agreement) specifically prohibit it, and U.S. courts have ruled that if such terms are present, they override the copyright law which expressly permits it (see Bowers v. Baystate Technologies). Sec. 103(f) of the DMCA (17 U.S.C. § 1201 (f)) says that a person who is in legal possession of a program, is permitted to reverse-engineer and circumvent its protection if this is necessary in order to achieve "interoperability" — a term broadly covering other devices and programs being able to interact with it, make use of it, and to use and transfer data to and from it, in useful ways. A limited exemption exists that allows the knowledge thus gained to be shared and used for interoperability purposes.
The unauthorised reproduction, translation, adaptation or transformation of the form of the code in which a copy of a computer program has been made available constitutes an infringement of the exclusive rights of the author. Nevertheless, circumstances may exist when such a reproduction of the code and translation of its form are indispensable to obtain the necessary information to achieve the interoperability of an independently created program with other programs. It has therefore to be considered that, in these limited circumstances only, performance of the acts of reproduction and translation by or on behalf of a person having a right to use a copy of the program is legitimate and compatible with fair practice and must therefore be deemed not to require the authorisation of the rightholder. An objective of this exception is to make it possible to connect all components of a computer system, including those of different manufacturers, so that they can work together. Such an exception to the author's exclusive rights may not be used in a way which prejudices the legitimate interests of the rightholder or which conflicts with a normal exploitation of the program.
Apache Cayenne is an open source persistence framework licensed under the Apache License, providing object-relational mapping (ORM) and remoting services. Cayenne binds one or more database schemas directly to Java objects, managing atomic commit and rollbacks, SQL generation, joins, sequences, and more. With Cayenne's Remote Object Persistence, those Java objects can even be persisted out to clients via Web Services. Or, with native XML serialization, objects can be further persisted to non-Java clients—such as an Ajax-capable browser.
Cayenne supports database reverse engineering and generation, as well as a Velocity-based class generation engine. All of these functions can be controlled directly through the CayenneModeler, a fully functional GUI tool. No XML- or annotation-based configuration is required. An entire database schema can be mapped directly to Java objects quickly, all from the comfort of the GUI-based CayenneModeler. These things together make Cayenne interesting for a user who is new to the enterprise world.
Cayenne supports other features, including caching, a complete object query syntax, relationship pre-fetching, on-demand object and relationship faulting, object inheritance, database auto-detection, and generic persisted objects. Most importantly, Cayenne can scale up or down to virtually any project size.Atari Games Corp. v. Nintendo of America Inc.
Atari Games Corp. v. Nintendo of America Inc., 975 F.2d 832 (Fed. Cir. 1992), is a United States Court of Appeals for the Federal Circuit case, in which the court held that Atari Games engaged in copyright infringement by copying Nintendo's lock-out system, the 10NES. The 10NES was designed to prevent Nintendo's video game console, the Nintendo Entertainment System (NES), from accepting unauthorized game cartridges. Atari, after unsuccessful attempts to reverse engineer the lock-out system, obtained an unauthorized copy of the source code from the Copyright Office and used it to create its 10NES replica, the Rabbit. The case involved copyright infringement claims by Nintendo and a defense based on fair use and copyright misuse by Atari.The Federal Circuit declared that Nintendo was likely to prove that Atari made unauthorized verbatim copies of the 10NES program and affirmed the imposition of a preliminary injunction by the United States District Court for the Northern District of California, preventing Atari from exploiting Nintendo's copyrighted code. The Federal Circuit, however, disagreed with the district court on the matter of reverse engineering and held that the intermediate copying of the 10NES object code incurred during the examination of the microchip itself was fair use. Atari's copyright misuse defense was precluded by the doctrine of unclean hands, as Atari had lied to the Copyright Office to obtain an unauthorized copy of the 10NES.Chiasmus (cipher)
Chiasmus is a secret German government block cipher that was leaked by reverse engineering. It became notorious for its dilettant use in the BSI's software GSTOOL, which used it in insecure Electronic Codebook (ECB) mode and generated the key with a pseudo random number generator initialized to the current system time, which means an effective key length that can easily be broken by brute force. The BSI tried to prevent the publication of these findings with legal threats.
Chiasmus seems to be resistant against the most common cryptographic attacks (linear and differential cryptanalysis), but a lot slower than the openly available state of the art ciphers such as AES.Chinese wall
Chinese wall is a business term describing an information barrier within an organization that was erected to prevent exchanges or communication that could lead to conflicts of interest. For example, a Chinese wall may be erected to separate and isolate people who make investments from those who are privy to confidential information that could improperly influence the investment decisions. Firms are generally required by law to safeguard insider information and ensure that improper trading does not occur.Comparison of database tools
The following tables compare general and technical information for a number of available database administrator tools. Please see individual product articles for further information. This article is neither all-inclusive nor necessarily up to date.Comparison of open-source wireless drivers
Wireless network cards for computers require control software to make them function (firmware, device drivers). This is a list of the status of some open-source drivers for 802.11 wireless network cards.Decompiler
A decompiler is a computer program that takes an executable file as input, and attempts to create a high level source file which can be recompiled successfully. It is therefore the opposite of a compiler, which takes a source file and makes an executable. Decompilers are usually unable to perfectly reconstruct the original source code, and as such, will frequently produce obfuscated code. Nonetheless, decompilers remain an important tool in the reverse engineering of computer software.Defence Science and Technology Organization
The Defence Science and Technology Organisation (reporting name: DESTO), is a multi-disciplinary programme agency under the Ministry of Defence Production, dedicated for evaluation of science and technology for use by the military.Its Director-General is a serving two-star general officer of the Pakistan Army, and is located at Chaklala cantonment. However, as an inter-services organisation it is overseen by the Joint Services Headquarters, and works closely with the Strategic Plans Division (SPD).
Created in 1963 in the Ministry of Defence, DESTO's clandestine work included in reverse engineering of the foreign technology and to avoid technological surprise from India. A limited amount knowledge of DESTO's project is available and much of its work is kept under secrecy.Among its various responsibilities it is Pakistan's national centre of expertise in chemical and biological defence.Disassembler
A disassembler is a computer program that translates machine language into assembly language—the inverse operation to that of an assembler. A disassembler differs from a decompiler, which targets a high-level language rather than an assembly language. Disassembly, the output of a disassembler, is often formatted for human creativity in the code writing process.
On CISC platforms with variable-width instructions, more than one disassembly may be correct. Disassemblers do not handle code that varies during execution.Fravia
Francesco Vianello (30 August 1952 – 3 May 2009), better known by his nickname Fravia (sometimes +Fravia or Fravia+), was a software reverse engineer, and hacker, known for his web archive of reverse engineering techniques and papers. He is also known for his work on steganography. He had taught on subjects such as data mining, anonymity, stalking, klebing, advertisement reversing and ad-busting.Fravia spoke six languages (including Latin) and had a degree in the history of the early Middle Ages. He was an expert in linguistics-related informatics. For five years he made available a large quantity of material related to reverse engineering through his website, which also hosted the advice of reverse engineering experts, known as reversers, who provided tutorials and essays on how to hack software code as well as advice related to the assembly and disassembly of applications, and software protection reversing.Fravia was a professor at the High Cracking University (+HCU), founded by Old Red Cracker (+ORC), a legendary figure in reverse engineering, to conduct research into Reverse Code Engineering. The addition of the "+" sign in front of the nickname of a reverser signified membership in the +HCU. His website was known as "+Fravia's Pages of Reverse Engineering" and he used it to challenge programmers as well as the wider society to "reverse engineer" the "brainwashing of a corrupt and rampant materialism". In its heyday, his website was receiving millions of visitors per year and its influence was "widespread".His web presence dates from 1995 when he first got involved in research related to reverse code engineering (RCE). In 2000 he changed his focus and concentrated on advanced internet search methods and the reverse engineering of search engine code.His websites "www.fravia.com" and "www.searchlores.org" contained a large amount of specialised information related to data mining. His website "www.searchlores.org" has been called a "very useful instrument for searching the web", and his "www.fravia.com" site has been described as "required reading for any spy wanting to go beyond simple Google searches."Industrial computed tomography
Industrial computed tomography (CT) scanning is any computer-aided tomographic process, usually X-ray computed tomography, that uses irradiation to produce three-dimensional internal and external representations of a scanned object. Industrial CT scanning has been used in many areas of industry for internal inspection of components. Some of the key uses for industrial CT scanning have been flaw detection, failure analysis, metrology, assembly analysis and reverse engineering applications. Just as in medical imaging, industrial imaging includes both nontomographic radiography (industrial radiography) and computed tomographic radiography (computed tomography).Intellectual property in China
Intellectual property rights (IPRs) have been acknowledged and protected in the People's Republic of China since the 1980s. The People's Republic of China has acceded to the major international conventions on protection of rights to intellectual property. Domestically, protection of intellectual property law has also been established by government legislation, administrative regulations, and decrees in the areas of trademark, copyright, and patent. This has led to the creation of a comprehensive legal framework to protect both local and foreign intellectual property. Despite this, copyright violations are common in the PRC, The American Chamber of Commerce in China surveyed over 500 of its members doing business in China regarding IPR for its 2016 China Business Climate Survey Report, and found that IPR enforcement is improving, but significant challenges still remain. The results show that the laws in place exceed their actual enforcement, with patent protection receiving the highest approval rate, while protection of trade secrets lags far behind.Proprietary protocol
In telecommunications, a proprietary protocol is a communications protocol owned by a single organization or individual.QuickTime Animation
QuickTime Animation format (also known as QuickTime RLE) is a video compression format and codec created by Apple Computer to enable playback of RGB video in real time without expensive hardware. It is generally found in the QuickTime container with the FourCC 'rle '. It can perform either lossless or lossy compression and is one of the few video codecs that supports an alpha channel. Supported color depths are 1-bit (monochrome), 15-bit RGB, 24-bit RGB, 32-bit ARGB, as well as palettized RGB. As a result of reverse-engineering of the format, a decoder is implemented in XAnim as well as an encoder and decoder in libavcodec.Sage Bionetworks
Sage Bionetworks is a nonprofit organization in Seattle that promotes open science and patient engagement in the research process. It is led by Lara Mangravite. It was co-founded by Stephen Friend and Eric Schadt.Sega v. Accolade
Sega Enterprises Ltd. v. Accolade, Inc., 977 F.2d 1510 (9th Cir. 1992), is a case in which the United States Court of Appeals for the Ninth Circuit applied American intellectual property law to the reverse engineering of computer software. Stemming from the publishing of several Sega Genesis games by video game publisher Accolade, which had disassembled Genesis software in order to publish games without being licensed by Sega, the case involved several overlapping issues, including the scope of copyright, permissible uses for trademarks, and the scope of the fair use doctrine for computer code.The case was filed in the U.S. District Court for the Northern District of California, which ruled in favor of Sega and issued an injunction against Accolade preventing them from publishing any more games for the Genesis and requiring them to recall all the existing Genesis games they had for sale. Accolade appealed the decision to the Ninth Circuit on the grounds that their reverse engineering of the Genesis was protected under fair use. The Ninth Circuit reversed the district court's order and ruled that Accolade's use of reverse engineering to publish Genesis titles was protected under fair use, and that its alleged violation of Sega trademarks was the fault of Sega. The case is frequently cited in matters involving reverse engineering and fair use under copyright law.Software cracking
Software cracking (known as "breaking" in the 1980s) is the modification of software to remove or disable features which are considered undesirable by the person cracking the software, especially copy protection features (including protection against the manipulation of software, serial number, hardware key, date checks and disc check) or software annoyances like nag screens and adware.
A crack refers to the means of achieving, for example a stolen serial number or a tool that performs that act of cracking. Some of these tools are called keygen, patch, or loader. A keygen is a handmade product serial number generator that often offers the ability to generate working serial numbers in your own name. A patch is a small computer program that modifies the machine code of another program. This has the advantage for a cracker to not include a large executable in a release when only a few bytes are changed. A loader modifies the startup flow of a program and does not remove the protection but circumvents it. A well-known example of a loader is a trainer used to cheat in games. Fairlight pointed out in one of their .nfo files that these type of cracks are not allowed for warez scene game releases. A nukewar has shown that the protection may not kick in at any point for it to be a valid crack.The distribution of cracked copies is illegal in most countries. There have been lawsuits over cracking software. It might be legal to use cracked software in certain circumstances. Educational resources for reverse engineering and software cracking are, however, legal and available in the form of Crackme programs.Strip (Unix)
In Unix and Unix-like operating systems, the strip program removes inessential information from executable binary programs and object files, thus potentially resulting in better performance and sometimes significantly less disk space usage ('inessential information' means information that is not required for correct functioning of the binary in normal execution). This information may consist of debugging and symbol information; however the standard leaves the scope of changes up to the implementer.
Furthermore, the use of strip can improve the security of the binary against reverse engineering. It will be more difficult to analyze a binary without its information and object's names.
The effect of strip can be achieved directly by the linker. For instance, in GNU Compiler Collection this option is "-s".
The GNU Project ships an implementation of strip as part of the GNU Binutils package. strip has been ported to other operating systems including Microsoft Windows.UML tool
A UML tool or Unified Modeling Language tool is a software application that supports some or all of the notation and semantics associated with the Unified Modeling Language (UML), which is the industry standard general-purpose modeling language for software engineering.
UML tool is used broadly here to include application programs which are not exclusively focused on UML, but which support some functions of the Unified Modeling Language, either as an add-on, as a component or as a part of their overall functionality.