TY - BOOK AU - Eilam,Eldad AU - Chikofsky,Elliot J. TI - Reversing: secrets of reverse engineering SN - 0764574817 (pbk.) AV - 0764574817 .E35 2005 U1 - 005.1 22 PY - 2005/// CY - Indianapolis, IN PB - Wiley KW - Software engineering KW - Reverse engineering KW - Computer security N1 - Foreword vii Acknowledgments xi Introduction xxiii Part I Reversing 101 1 Chapter 1 Foundations 3 What Is Reverse Engineering? 3 Software Reverse Engineering: Reversing 4 Reversing Applications 4 Security-Related Reversing 5 Malicious Software 5 Reversing Cryptographic Algorithms 6 Digital Rights Management 7 Auditing Program Binaries 7 Reversing in Software Development 8 Achieving Interoperability with Proprietary Software 8 Developing Competing Software 8 Evaluating Software Quality and Robustness 9 Low-Level Software 9 Assembly Language 10 Compilers 11 Virtual Machines and Bytecodes 12 Operating Systems 13 Contents xiii The Reversing Process 13 System-Level Reversing 14 Code-Level Reversing 14 The Tools 14 System-Monitoring Tools 15 Disassemblers 15 Debuggers 15 Decompilers 16 Is Reversing Legal? 17 Interoperability 17 Competition 18 Copyright Law 19 Trade Secrets and Patents 20 The Digital Millenium Copyright Act 20 DMCACases 22 License Agreement Considerations 23 Code Samples & Tools 23 Conclusion 23 Chapter 2 Low-Level Software 25 High-Level Perspectives 26 Program Structure 26 Modules 28 Common Code Constructs 28 Data Management 29 Variables 30 User-Defined Data Structures 30 Lists 31 Control Flow 32 High-Level Languages 33 C 34 C++ 35 Java 36 C# 36 Low-Level Perspectives 37 Low-Level Data Management 37 Registers 39 The Stack 40 Heaps 42 Executable Data Sections 43 Control Flow 43 Assembly Language 101 44 Registers 44 Flags 46 Instruction Format 47 Basic Instructions 48 Moving Data 49 Arithmetic 49 Comparing Operands 50 xiv Contents Conditional Branches 51 Function Calls 51 Examples 52 A Primer on Compilers and Compilation 53 Defining a Compiler 54 Compiler Architecture 55 Front End 55 Intermediate Representations 55 Optimizer 56 Back End 57 Listing Files 58 Specific Compilers 59 Execution Environments 60 Software Execution Environments (Virtual Machines) 60 Bytecodes 61 Interpreters 61 Just-in-Time Compilers 62 Reversing Strategies 62 Hardware Execution Environments in Modern Processors 63 Intel NetBurst 65 μops (Micro-Ops) 65 Pipelines 65 Branch Prediction 67 Conclusion 68 Chapter 3 Windows Fundamentals 69 Components and Basic Architecture 70 Brief History 70 Features 70 Supported Hardware 71 Memory Management 71 Virtual Memory and Paging 72 Paging 73 Page Faults 73 Working Sets 74 Kernel Memory and User Memory 74 The Kernel Memory Space 75 Section Objects 77 VAD Trees 78 User-Mode Allocations 78 Memory Management APIs 79 Objects and Handles 80 Named objects 81 Processes and Threads 83 Processes 84 Threads 84 Context Switching 85 Synchronization Objects 86 Process Initialization Sequence 87 Contents xv Application Programming Interfaces 88 The Win32 API 88 The Native API 90 System Calling Mechanism 91 Executable Formats 93 Basic Concepts 93 Image Sections 95 Section Alignment 95 Dynamically Linked Libraries 96 Headers 97 Imports and Exports 99 Directories 99 Input and Output 103 The I/O System 103 The Win32 Subsystem 104 Object Management 105 Structured Exception Handling 105 Conclusion 107 Chapter 4 Reversing Tools 109 Different Reversing Approaches 110 Offline Code Analysis (Dead-Listing) 110 Live Code Analysis 110 Disassemblers 110 IDA Pro 112 ILDasm 115 Debuggers 116 User-Mode Debuggers 118 OllyDbg 118 User Debugging in WinDbg 119 IDA Pro 121 PEBrowse Professional Interactive 122 Kernel-Mode Debuggers 122 Kernel Debugging in WinDbg 123 Numega SoftICE 124 Kernel Debugging on Virtual Machines 127 Decompilers 129 System-Monitoring Tools 129 Patching Tools 131 Hex Workshop 131 Miscellaneous Reversing Tools 133 Executable-Dumping Tools 133 DUMPBIN 133 PEView 137 PEBrowse Professional 137 Conclusion 138 xvi Contents Part II Applied Reversing 139 Chapter 5 Beyond the Documentation 141 Reversing and Interoperability 142 Laying the Ground Rules 142 Locating Undocumented APIs 143 What Are We Looking For? 144 Case Study: The Generic Table API in NTDLL.DLL 145 RtlInitializeGenericTable 146 RtlNumberGenericTableElements 151 RtlIsGenericTableEmpty 152 RtlGetElementGenericTable 153 Setup and Initialization 155 Logic and Structure 159 Search Loop 1 161 Search Loop 2 163 Search Loop 3 164 Search Loop 4 165 Reconstructing the Source Code 165 RtlInsertElementGenericTable 168 RtlLocateNodeGenericTable 170 RtlRealInsertElementWorker 178 Splay Trees 187 RtlLookupElementGenericTable 188 RtlDeleteElementGenericTable 193 Putting the Pieces Together 194 Conclusion 196 Chapter 6 Deciphering File Formats 199 Cryptex 200 Using Cryptex 201 Reversing Cryptex 202 The Password Verification Process 207 Catching the “Bad Password” Message 207 The Password Transformation Algorithm 210 Hashing the Password 213 The Directory Layout 218 Analyzing the Directory Processing Code 218 Analyzing a File Entry 223 Dumping the Directory Layout 227 The File Extraction Process 228 Scanning the File List 234 Decrypting the File 235 The Floating-Point Sequence 236 The Decryption Loop 238 Verifying the Hash Value 239 The Big Picture 239 Digging Deeper 241 Conclusion 242 Contents xvii Chapter 7 Auditing Program Binaries 243 Defining the Problem 243 Vulnerabilities 245 Stack Overflows 245 A Simple Stack Vulnerability 247 Intrinsic Implementations 249 Stack Checking 250 Nonexecutable Memory 254 Heap Overflows 255 String Filters 256 Integer Overflows 256 Arithmetic Operations on User-Supplied Integers 258 Type Conversion Errors 260 Case-Study: The IIS Indexing Service Vulnerability 262 CVariableSet::AddExtensionControlBlock 263 DecodeURLEscapes 267 Conclusion 271 Chapter 8 Reversing Malware 273 Types of Malware 274 Viruses 274 Worms 274 Trojan Horses 275 Backdoors 276 Mobile Code 276 Adware/Spyware 276 Sticky Software 277 Future Malware 278 Information-Stealing Worms 278 BIOS/Firmware Malware 279 Uses of Malware 280 Malware Vulnerability 281 Polymorphism 282 Metamorphism 283 Establishing a Secure Environment 285 The Backdoor.Hacarmy.D 285 Unpacking the Executable 286 Initial Impressions 290 The Initial Installation 291 Initializing Communications 294 Connecting to the Server 296 Joining the Channel 298 Communicating with the Backdoor 299 Running SOCKS4 Servers 303 Clearing the Crime Scene 303 The Backdoor.Hacarmy.D: A Command Reference 304 Conclusion 306 xviii Contents Part III Cracking 307 Chapter 9 Piracy and Copy Protection 309 Copyrights in the New World 309 The Social Aspect 310 Software Piracy 310 Defining the Problem 311 Class Breaks 312 Requirements 313 The Theoretically Uncrackable Model 314 Types of Protection 314 Media-Based Protections 314 Serial Numbers 315 Challenge Response and Online Activations 315 Hardware-Based Protections 316 Software as a Service 317 Advanced Protection Concepts 318 Crypto-Processors 318 Digital Rights Management 319 DRM Models 320 The Windows Media Rights Manager 321 Secure Audio Path 321 Watermarking 321 Trusted Computing 322 Attacking Copy Protection Technologies 324 Conclusion 324 Chapter 10 Antireversing Techniques 327 Why Antireversing? 327 Basic Approaches to Antireversing 328 Eliminating Symbolic Information 329 Code Encryption 330 Active Antidebugger Techniques 331 Debugger Basics 331 The IsDebuggerPresent API 332 SystemKernelDebuggerInformation 333 Detecting SoftICE Using the Single-Step Interrupt 334 The Trap Flag 335 Code Checksums 335 Confusing Disassemblers 336 Linear Sweep Disassemblers 337 Recursive Traversal Disassemblers 338 Applications 343 Code Obfuscation 344 Control Flow Transformations 346 Opaque Predicates 346 Confusing Decompilers 348 Table Interpretation 348 Contents xix Inlining and Outlining 353 Interleaving Code 354 Ordering Transformations 355 Data Transformations 355 Modifying Variable Encoding 355 Restructuring Arrays 356 Conclusion 356 Chapter 11 Breaking Protections 357 Patching 358 Keygenning 364 Ripping Key-Generation Algorithms 365 Advanced Cracking: Defender 370 Reversing Defender’s Initialization Routine 377 Analyzing the Decrypted Code 387 SoftICE’s Disappearance 396 Reversing the Secondary Thread 396 Defeating the “Killer” Thread 399 Loading KERNEL32.DLL 400 Reencrypting the Function 401 Back at the Entry Point 402 Parsing the Program Parameters 404 Processing the Username 406 Validating User Information 407 Unlocking the Code 409 Brute-Forcing Your Way through Defender 409 Protection Technologies in Defender 415 Localized Function-Level Encryption 415 Relatively Strong Cipher Block Chaining 415 Reencrypting 416 Obfuscated Application/Operating System Interface 416 Processor Time-Stamp Verification Thread 417 Runtime Generation of Decryption Keys 418 Interdependent Keys 418 User-Input-Based Decryption Keys 419 Heavy Inlining 419 Conclusion 419 Part IV Beyond Disassembly 421 Chapter 12 Reversing .NET 423 Ground Rules 424 .NET Basics 426 Managed Code 426 .NET Programming Languages 428 Common Type System (CTS) 428 Intermediate Language (IL) 429 The Evaluation Stack 430 Activation Records 430 xx Contents IL Instructions 430 IL Code Samples 433 Counting Items 433 A Linked List Sample 436 Decompilers 443 Obfuscators 444 Renaming Symbols 444 Control Flow Obfuscation 444 Breaking Decompilation and Disassembly 444 Reversing Obfuscated Code 445 XenoCode Obfuscator 446 DotFuscator by Preemptive Solutions 448 Remotesoft Obfuscator and Linker 451 Remotesoft Protector 452 Precompiled Assemblies 453 Encrypted Assemblies 453 Conclusion 455 Chapter 13 Decompilation 457 Native Code Decompilation: An Unsolvable Problem? 457 Typical Decompiler Architecture 459 Intermediate Representations 459 Expressions and Expression Trees 461 Control Flow Graphs 462 The Front End 463 Semantic Analysis 463 Generating Control Flow Graphs 464 Code Analysis 466 Data-Flow Analysis 466 Single Static Assignment (SSA) 467 Data Propagation 468 Register Variable Identification 470 Data Type Propagation 471 Type Analysis 472 Primitive Data Types 472 Complex Data Types 473 Control Flow Analysis 475 Finding Library Functions 475 The Back End 476 Real-World IA-32 Decompilation 477 Conclusion 477 UR - http://www.loc.gov/catdir/enhancements/fy0628/2005921595-b.html UR - https://www.pdfdrive.com/reverse-engineering-d18717877.html UR - https://www.pdfdrive.com/reverse-engineering-d18717877.html ER -