Background Image
Table of Contents Table of Contents
Development with Python Next Page
Information
Show Menu
Development with Python Next Page

 


Cover 		1
Copyright 		4
Table of Contents 		5
Preface 		15
Why I Wrote a Book About Test-Driven Development 		15
Aims of This Book 		17
Outline 		17
Conventions Used in This Book 		18
Using Code Examples 		19
Safari® Books Online 		19
Contacting O’Reilly 		20
Prerequisites and Assumptions 		21
Python 3 and Programming 		21
How HTML Works 		22
JavaScript 		22
Required Software Installations 		22
Git’s Default Editor, and Other Basic Git Config 		25
Required Python Modules 		25
Acknowledgments 		27
Part I. The Basics of TDD and Django 		29
Chapter 1. Getting Django Set Up Using a Functional Test 		31
Obey the Testing Goat! Do Nothing Until You Have a Test 		31
Getting Django Up and Running 		34
Starting a Git Repository 		36
Chapter 2. Extending Our Functional Test Using the unittest Module 		41
Using a Functional Test to Scope Out a Minimum Viable App 		41
The Python Standard Library’s unittest Module 		44
Implicit waits 		46
Commit 		46
Chapter 3. Testing a Simple Home Page with Unit Tests 		49
Our First Django App, and Our First Unit Test 		50
Unit Tests, and How They Differ from Functional Tests 		50
Unit Testing in Django 		51
Django’s MVC, URLs, and View Functions 		52
At Last! We Actually Write Some Application Code! 		54
urls.py 		55
Unit Testing a View 		58
The Unit-Test/Code Cycle 		59
Chapter 4. What Are We Doing with All These Tests? 		63
Programming Is like Pulling a Bucket of Water up from a Well 		64
Using Selenium to Test User Interactions 		65
The “Don’t Test Constants” Rule, and Templates to the Rescue 		68
Refactoring to Use a Template 		68
On Refactoring 		72
A Little More of Our Front Page 		73
Recap: The TDD Process 		75
Chapter 5. Saving User Input 		79
Wiring Up Our Form to Send a POST Request 		79
Processing a POST Request on the Server 		82
Passing Python Variables to Be Rendered in the Template 		83
Three Strikes and Refactor 		87
The Django ORM and Our First Model 		88
Our First Database Migration 		90
The Test Gets Surprisingly Far 		91
A New Field Means a New Migration 		92
Saving the POST to the Database 		93
Redirect After a POST 		96
Better Unit Testing Practice: Each Test Should Test One Thing 		96
Rendering Items in the Template 		97
Creating Our Production Database with migrate 		99
Chapter 6. Getting to the Minimum Viable Site 		105
Ensuring Test Isolation in Functional Tests 		105
Running Just the Unit Tests 		108
Small Design When Necessary 		109
YAGNI! 		110
REST 		110
Implementing the New Design Using TDD 		111
Iterating Towards the New Design 		114
Testing Views, Templates, and URLs Together with the Django Test Client 		115
A New Test Class 		116
A New URL 		116
A New View Function 		117
A Separate Template for Viewing Lists 		118
Another URL and View for Adding List Items 		120
A Test Class for New List Creation 		121
A URL and View for New List Creation 		122
Removing Now-Redundant Code and Tests 		123
Pointing Our Forms at the New URL 		124
Adjusting Our Models 		125
A Foreign Key Relationship 		127
Adjusting the Rest of the World to Our New Models 		128
Each List Should Have Its Own URL 		130
Capturing Parameters from URLs 		131
Adjusting new_list to the New World 		132
One More View to Handle Adding Items to an Existing List 		133
Beware of Greedy Regular Expressions! 		134
The Last New URL 		134
The Last New View 		135
But How to Use That URL in the Form? 		136
A Final Refactor Using URL includes 		138
Part II. Web Development Sine Qua Nons 		141
Chapter 7. Prettification: Layout and Styling, and What to Test About It 		143
What to Functionally Test About Layout and Style 		143
Prettification: Using a CSS Framework 		146
Django Template Inheritance 		148
Integrating Bootstrap 		149
Rows and Columns 		150
Static Files in Django 		150
Switching to StaticLiveServerCase 		152
Using Bootstrap Components to Improve the Look of the Site 		153
Jumbotron! 		153
Large Inputs 		153
Table Styling 		154
Using Our Own CSS 		154
What We Glossed Over: collectstatic and Other Static Directories 		155
A Few Things That Didn’t Make It 		158
Chapter 8. Testing Deployment Using a Staging Site 		159
TDD and the Danger Areas of Deployment 		160
As Always, Start with a Test 		161
Getting a Domain Name 		163
Manually Provisioning a Server to Host Our Site 		164
Choosing Where to Host Our Site 		164
Spinning Up a Server 		165
User Accounts, SSH, and Privileges 		165
Installing Nginx 		166
Configuring Domains for Staging and Live 		167
Using the FT to Confirm the Domain Works and Nginx Is Running 		167
Deploying Our Code Manually 		168
Adjusting the Database Location 		169
Creating a Virtualenv 		170
Simple Nginx Configuration 		172
Creating the Database with migrate 		175
Getting to a Production-Ready Deployment 		176
Switching to Gunicorn 		176
Getting Nginx to Serve Static Files 		177
Switching to Using Unix Sockets 		178
Switching DEBUG to False and Setting ALLOWED_HOSTS 		179
Using Upstart to Make Sure Gunicorn Starts on Boot 		179
Saving Our Changes: Adding Gunicorn to Our requirements.txt 		180
Automating 		180
“Saving Your Progress” 		184
Chapter 9. Automating Deployment with Fabric 		185
Breakdown of a Fabric Script for Our Deployment 		186
Trying It Out 		190
Deploying to Live 		191
Nginx and Gunicorn Config Using sed 		193
Git Tag the Release 		194
Further Reading 		194
Chapter 10. Input Validation and Test Organisation 		197
Validation FT: Preventing Blank Items 		197
Skipping a Test 		198
Splitting Functional Tests out into Many Files 		199
Running a Single Test File 		202
Fleshing Out the FT 		202
Using Model-Layer Validation 		203
Refactoring Unit Tests into Several Files 		203
Unit Testing Model Validation and the self.assertRaises Context Manager 		205
A Django Quirk: Model Save Doesn’t Run Validation 		206
Surfacing Model Validation Errors in the View 		206
Checking Invalid Input Isn’t Saved to the Database 		209
Django Pattern: Processing POST Requests in the Same View as Renders the Form 		211
Refactor: Transferring the new_item Functionality into view_list 		212
Enforcing Model Validation in view_list 		214
Refactor: Removing Hardcoded URLs 		215
The {% url %} Template Tag 		216
Using get_absolute_url for Redirects 		216
Chapter 11. A Simple Form 		221
Moving Validation Logic into a Form 		221
Exploring the Forms API with a Unit Test 		222
Switching to a Django ModelForm 		223
Testing and Customising Form Validation 		224
Using the Form in Our Views 		226
Using the Form in a View with a GET Request 		226
A Big Find and Replace 		229
Using the Form in a View That Takes POST Requests 		231
Adapting the Unit Tests for the new_list View 		231
Using the Form in the View 		232
Using the Form to Display Errors in the Template 		233
Using the Form in the Other View 		233
A Helper Method for Several Short Tests 		234
Using the Form’s Own Save Method 		236
Chapter 12. More Advanced Forms 		239
Another FT for Duplicate Items 		239
Preventing Duplicates at the Model Layer 		240
A Little Digression on Queryset Ordering and String Representations 		242
Rewriting the Old Model Test 		244
Some Integrity Errors Do Show Up on Save 		245
Experimenting with Duplicate Item Validation at the Views Layer 		246
A More Complex Form to Handle Uniqueness Validation 		247
Using the Existing List Item Form in the List View 		249
Chapter 13. Dipping Our Toes, Very Tentatively, into JavaScript 		253
Starting with an FT 		253
Setting Up a Basic JavaScript Test Runner 		254
Using jQuery and the Fixtures Div 		257
Building a JavaScript Unit Test for Our Desired Functionality 		260
Javascript Testing in the TDD Cycle 		262
Columbo Says: Onload Boilerplate and Namespacing 		262
A Few Things That Didn’t Make It 		263
Chapter 14. Deploying Our New Code 		265
Staging Deploy 		265
Live Deploy 		265
What to Do If You See a Database Error 		266
Wrap-Up: git tag the New Release 		266
Part III. More Advanced Topics 		267
Chapter 15. User Authentication, Integrating Third-Party Plugins, and Mocking with JavaScript 		269
Mozilla Persona (BrowserID) 		270
Exploratory Coding, aka “Spiking” 		270
Starting a Branch for the Spike 		271
Frontend and JavaScript Code 		271
The Browser-ID Protocol 		272
The Server Side: Custom Authentication 		273
De-spiking 		279
A Common Selenium Technique: Explicit Waits 		281
Reverting Our Spiked Code 		283
JavaScript Unit Tests Involving External Components: Our First Mocks! 		284
Housekeeping: A Site-Wide Static Files Folder 		284
Mocking: Who, Why, What? 		285
Namespacing 		286
A Simple Mock to Unit Tests Our initialize Function 		286
More Advanced Mocking 		292
Checking Call Arguments 		295
QUnit setup and teardown, Testing Ajax 		296
More Nested Callbacks! Testing Asynchronous Code 		300
Chapter 16. Server-Side Authentication and Mocking in Python 		305
A Look at Our Spiked Login View 		305
Mocking in Python 		306
Testing Our View by Mocking Out authenticate 		306
Checking the View Actually Logs the User In 		309
De-spiking Our Custom Authentication Backend: Mocking Out an Internet Request 		313
1 if = 1 More Test 		314
Patching at the Class Level 		315
Beware of Mocks in Boolean Comparisons 		318
Creating a User if Necessary 		319
The get_user Method 		319
A Minimal Custom User Model 		321
A Slight Disappointment 		323
Tests as Documentation 		324
Users Are Authenticated 		325
The Moment of Truth: Will the FT Pass? 		326
Finishing Off Our FT, Testing Logout 		327
Chapter 17. Test Fixtures, Logging, and Server-Side Debugging 		331
Skipping the Login Process by Pre-creating a Session 		331
Checking It Works 		333
The Proof Is in the Pudding: Using Staging to Catch Final Bugs 		334
Setting Up Logging 		335
Fixing the Persona Bug 		337
Managing the Test Database on Staging 		339
A Django Management Command to Create Sessions 		339
Getting the FT to Run the Management Command on the Server 		340
An Additional Hop via subprocess 		342
Baking In Our Logging Code 		345
Using Hierarchical Logging Config 		346
Wrap-Up 		348
Chapter 18. Finishing “My Lists”: Outside-In TDD 		351
The Alternative: “Inside Out” 		351
Why Prefer “Outside-In”? 		351
The FT for “My Lists” 		352
The Outside Layer: Presentation and Templates 		353
Moving Down One Layer to View Functions (the Controller) 		354
Another Pass, Outside-In 		355
A Quick Restructure of the Template Inheritance Hierarchy 		355
Designing Our API Using the Template 		356
Moving Down to the Next Layer: What the View Passes to the Template 		357
The Next “Requirement” from the Views Layer: New Lists Should Record Owner 		358
A Decision Point: Whether to Proceed to the Next Layer with a Failing Test 		359
Moving Down to the Model Layer 		359
Final Step: Feeding Through the .name API from the Template 		361
Chapter 19. Test Isolation, and “Listening to Your Tests” 		365
Revisiting Our Decision Point: The Views Layer Depends on Unwritten Models Code 		365
A First Attempt at Using Mocks for Isolation 		366
Using Mock side_effects to Check the Sequence of Events 		367
Listen to Your Tests: Ugly Tests Signal a Need to Refactor 		369
Rewriting Our Tests for the View to Be Fully Isolated 		370
Keep the Old Integrated Test Suite Around as a Sanity Check 		370
A New Test Suite with Full Isolation 		371
Thinking in Terms of Collaborators 		371
Moving Down to the Forms Layer 		375
Keep Listening to Your Tests: Removing ORM Code from Our Application 		376
Finally, Moving Down to the Models Layer 		379
Back to Views 		381
The Moment of Truth (and the Risks of Mocking) 		382
Thinking of Interactions Between Layers as “Contracts” 		383
Identifying Implicit Contracts 		384
Fixing the Oversight 		385
One More Test 		386
Tidy Up: What to Keep from Our Integrated Test Suite 		387
Removing Redundant Code at the Forms Layer 		387
Removing the Old Implementation of the View 		388
Removing Redundant Code at the Forms Layer 		389
Conclusions: When to Write Isolated Versus Integrated Tests 		390
Let Complexity Be Your Guide 		391
Should You Do Both? 		391
Onwards! 		391
Chapter 20. Continuous Integration (CI) 		393
Installing Jenkins 		393
Configuring Jenkins Security 		395
Adding Required Plugins 		396
Setting Up Our Project 		397
First Build! 		399
Setting Up a Virtual Display so the FTs Can Run Headless 		400
Taking Screenshots 		402
A Common Selenium Problem: Race Conditions 		406
Running Our QUnit JavaScript Tests in Jenkins with PhantomJS 		409
Installing node 		410
Adding the Build Steps to Jenkins 		411
More Things to Do with a CI Server 		412
Chapter 21. The Token Social Bit, the Page Pattern, and an Exercise for the Reader 		415
An FT with Multiple Users, and addCleanup 		415
Implementing the Selenium Interact/Wait Pattern 		417
The Page Pattern 		418
Extend the FT to a Second User, and the “My Lists” Page 		421
An Exercise for the Reader 		423
Chapter 22. Fast Tests, Slow Tests, and Hot Lava 		425
Thesis: Unit Tests Are Superfast and Good Besides That 		426
Faster Tests Mean Faster Development 		426
The Holy Flow State 		427
Slow Tests Don’t Get Run as Often, Which Causes Bad Code 		427
We’re Fine Now, but Integrated Tests Get Slower Over Time 		427
Don’t Take It from Me 		427
And Unit Tests Drive Good Design 		428
The Problems with “Pure” Unit Tests 		428
Isolated Tests Can Be Harder to Read and Write 		428
Isolated Tests Don’t Automatically Test Integration 		428
Unit Tests Seldom Catch Unexpected Bugs 		428
Mocky Tests Can Become Closely Tied to Implementation 		428
But All These Problems Can Be Overcome 		429
Synthesis: What Do We Want from Our Tests, Anyway? 		429
Correctness 		429
Clean, Maintainable Code 		429
Productive Workflow 		430
Evaluate Your Tests Against the Benefits You Want from Them 		430
Architectural Solutions 		430
Ports and Adapters/Hexagonal/Clean Architecture 		431
Functional Core, Imperative Shell 		431
Conclusion 		432
Obey the Testing Goat! 		435
Appendix A. PythonAnywhere 		437
Running Firefox Selenium Sessions with Xvfb 		437
Setting Up Django as a PythonAnywhere Web App 		438
Cleaning Up /tmp 		439
Screenshots 		439
The Deployment Chapter 		439
Appendix B. Django Class-Based Views 		441
Class-Based Generic Views 		441
The Home Page as a FormView 		442
Using form_valid to Customise a CreateView 		443
A More Complex View to Handle Both Viewing and Adding to a List 		445
The Tests Guide Us, for a While 		445
Until We’re Left with Trial and Error 		446
Back on Track 		447
Is That Your Final Answer? 		448
Compare Old and New 		448
Best Practices for Unit Testing CBGVs? 		448
Take-Home: Having Multiple, Isolated View Tests with Single Assertions Helps 		449
Appendix C. Provisioning with Ansible 		451
Installing System Packages and Nginx 		451
Configuring Gunicorn, and Using Handlers to Restart Services 		453
What to Do Next 		454
Move Deployment out of Fabric and into Ansible 		454
Use Vagrant to Spin Up a Local VM 		454
Appendix D. Testing Database Migrations 		455
An Attempted Deploy to Staging 		455
Running a Test Migration Locally 		456
Entering Problematic Data 		456
Copying Test Data from the Live Site 		456
Confirming the Error 		457
Inserting a Data Migration 		457
Re-creating the Old Migration 		458
Testing the New Migrations Together 		458
Conclusions 		459
Appendix E. What to Do Next 		461
Notifications—Both on the Site and by Email 		461
Switch to Postgres 		461
Run Your Tests Against Different Browsers 		462
404 and 500 Tests 		462
The Django Admin Site 		462
Investigate a BDD Tool 		462
Write Some Security Tests 		463
Test for Graceful Degradation 		463
Caching and Performance Testing 		463
JavaScript MVC Frameworks 		463
Async and Websockets 		463
Switch to Using py.test 		464
Client-Side Encryption 		464
Your Suggestion Here 		464
Appendix F. Cheat Sheet 		465
Initial Project Setup 		465
The Basic TDD Workflow 		465
Moving Beyond dev-only Testing 		466
General Testing Best Practices 		467
Selenium/Functional Testing Best Practices 		467
Outside-In, Test Isolation Versus Integrated Tests, and Mocking 		468
Appendix G. Bibliography 		469
Index 		471
About the Author 		478
 FlippingBook