Why is Jenkins mainly related to DevOps

From ContinuousIntegration to ContinuousDelivery

From Continuous Integration to Continuous Delivery Orientation in Objects GmbH Weinheimer Str. 68 68309 Mannheim Steffen Schluff Version: 1.1 www.oio.de [email protected] Outline Introduction Continuous Delivery DevOps Summary 2 1

Outline Introduction Continuous Delivery DevOps Summary 3 Been there, done that (1) Build Build Tool Return Build Results CI Server Update VCS Publish Build Results Commit Notify CI Server Website with Build Results Development Team 4 2

Been there, done that (2) Daily Build and Smoke Tests are already old hat First publication by Steve McConnell in 1996 Topic was already known before Continuous Integration Article by Martin Fowler in 2000 Topic got a good name Definition Key Practices ( Automate the build, Make it self-testing,) First provision of ready-made tools (CruiseControl) Continuous integration is now part of the good tone Freedom of choice between various servers (Jenkins, Hudson, Bamboo,) Definition of best practices, patterns and anti-pattern problems of the second generation: test runtimes, virtualization, 5 developers Kosmos Subversive IDE Eclipse Mylyn Issue-Tracker Atlassian JIRA SVN Plugin View VC SCM Subversion JIRA Plugin SVN Plugin CI-Server Jenkins 6 3

Wasn't there anything else? (1) Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. First principle behind the Agile Manifesto (http://agilemanifesto.org/principles.html) 7 Was there something else? (2) Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. First principle behind the Agile Manifesto (http://agilemanifesto.org/principles.html) 8 4

Ha Ha Only Serious Subversive IDE Eclipse Mylyn ??? Issue-Tracker Atlassian JIRA Customer SVN Plugin View VC SCM Subversion JIRA Plugin SVN Plugin CI-Server Jenkins 9 You should come this far and no further Continuous integration focuses on developer CI server green, commit successful, developer happy But no provision for testing and No go-live Build Tool CI-Server VCS Developer CI-Server Results Dev Team Customer UAT Ops QA (http://www.public-domain-photos.com/free-cliparts) 10 5

Doesn't anyone think about the customer? Customer wants availability of its functionality No interest in whether CI server is red, green, yellow or blue Between a good CI build and customer availability, the release release step is often not as well controlled as the CI ecosystem release model Big Bang (aka the classic) Manual, time-consuming , Complicated, many people involved, error-prone 11 Don t do that then! Rare but large releases as a consequence Organizational avoidance reaction At the same time high stress factor with unplanned releases Little practice leads to high error rate with hotfixes Frustrating for the customer Even individual features seem to take a long time We don't want to frustrate the customer, something has to change here! 12 6

Good things take time Duration of a customer Idea to go live (Concept to Cash) Visualization as Value Stream Map Development and provision of Product Opportunities Assessment Product Definition Product Planning Development Final Test and Release Release 3 days 1 week 10 days 7 weeks 1 week 2 hours processing time ( value-adding) Waiting time (not value-adding) 1 week 10 days 3 days 5 days 2 days (According to Continuous Delivery / J. Humble, D. Farley) 13 Stay on target Change Change Time Time [] the ability to rapidly, reliably and repeatedly push out enhancements and bug fixes to customers at low risk and with minimal manual overhead. (http://en.wikipedia.org/wiki/continuous_delivery) 14 7

Buzzword (1) Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. First principle behind the Agile Manifesto (http://agilemanifesto.org/principles.html) 15 Buzzword (2) Our highest priority is to satisfy the customer through early and continuous delivery of valuable software. First principle behind the Agile Manifesto (http://agilemanifesto.org/principles.html) 16 8

Outline Introduction Continuous Delivery DevOps Summary 17 Continuous Delivery Continuous Delivery is not Continuous Integration. Continuous delivery is being in the position to ship your product whenever you want, day or night. (Neal Ford) Earlier use of the term and roots Agile Manifesto (2001) Deployment Pipeline (2004/2005) Book of the same name by Jez Humble & David Farley Actual term coining (2010) Focus on automation and collaboration 18 9

Continuous Delivery Core Thoughts Create a Repeatable, Reliable Process for Releasing Software If It Hurts, Do It More Frequently, and Bring the Pain Forward Automate Almost Everything Keep Everything in Version Control Everybody Is Responsible for the Delivery Process Done Means Released become? (According to Continuous Delivery / J. Humble, D. Farley) 19 Deployment Pipeline A first look Central abstraction Deployment Pipeline Visualization of all process parts for all involved Improved feedback during execution Possibility of a fully automatic release in all environments Commit Stage Acceptance Test Stage Performance Test Stage User acceptance Stage Productive Stage Compile Unit Execute Tests Packaging Execute Code Analysis Configure Environment Deploy Binaries Execute Smoke Tests Acceptance Execute Tests Configure Environment Deploy Binaries Execute Smoke Tests Execute Performance Tests Configure Environment Deploy Binaries Execute Smoke Tests Configure Environment Deploy Binaries Execute Smoke Tests (After Continuous Delivery / J. Humble, D. Farley) 20 10

Humble, D. Farley) 22 11

Deployment Pipeline A second look Source code Infrastructure and application configuration VCS Quality assurance: Self-directed deployment User acceptance Stage commit Stage Acceptance test stage Productive stage automated: one-time creation of the artifacts and release in the artifact repository Application operation: Release at the push of a button Performance test stage Application operation: Release at the push of a button Artifact repository (according to Continuous Delivery / J. Humble, D. Farley) 23 Continuous Delivery Principles & Methods (1) Continuous optimization The responsibility of all parties involved (Development, Operations,) Artifacts are built once and managed in an artifact repository and all stages made available The goal is identical deployment in all environments Environment specifics through own configurations Configuration management Basis for one-time created artifacts Includes software and infrastructure (Infrastructure as code) Configurations are versioned 24 12

Continuous Delivery Principles & Methods (2) Automation As extensive as possible Also includes all aspects of the infrastructure (including OS) Characterized by development and operations tests Basis for automation and pipeline processing Provide security for successful changes Smoke tests especially for deployment monitoring Enables feedback for operating personnel Basis for ongoing optimization 25 All theory is gray Success factors of Continuous Integration in retrospect were catchy name Concrete key practices Applicable tools For success Continuous delivery is still missing a good tool First impulse often self-made solutions (home grown), but often quickly outdated with poor costs Benefit ratio Every project has its own specialties in practice Web App vs. Mobile vs. Rich Client, programming language, OS, etc. Thus, continuous delivery implementations are also different. So there is no continuous delivery tool at all? 26 13

Continuous Delivery Tooling (1) The deployment pipeline has its foundation in the process of continuous integration and is in essence the principles of continuous integration taken to its logical conclusion. (J. Humble, D. Farley) Previous CI Focus Commit Stage Acceptance Test Stage Performance Test Stage User Acceptance Stage Productive Stage 27 Continuous Delivery Tooling (2) CI servers are already used for all possible project types and integrate various tool types (build, Test, Lint, Coverage,) Tools for individual continuous delivery concepts are available. Artifact repositories (CI server own repos, Maven,) Infrastructure as code (Puppet, Chef,) Continuous integration becomes continuous delivery server through the integration of the new CD-specific tool types Provision of a deployment pipeline (including stages, jobs, triggers) 28 14

Names are bogus and smoke Every common CI server offers pipeline modules and is therefore a CD server. Specific names may vary Partly historical reasons, partly differentiation from the competition Exemplary examples (Your Mileage May Vary) Jenkins: Build Jobs, Build Steps, Post-build Actions, various plugins Go: Go Pipelines, Stages, Jobs, Tasks Bamboo: Build Plans, Stages, Jobs, Tasks The first logical step is visualization of the pipeline. Previous activities of a pipeline, ie past pipeline instances 29 Build Pipeline Plugin in Jenkins 30 15

Value Stream Maps in Go (http://www.thoughtworks.com/products/docs/go/13.3/help/) 31 Release details in Bamboo 32 16

Outline Introduction Continuous Delivery DevOps Summary 33 The Problem in Detail Continuous Delivery is a continuation of Continuous Integration Continuous Integration Tooling is known and established But: Now not only development involved, but also Quality Assurance in Stage Manual Testing Operations in Stage Release Problem is exacerbated by New methods and technologies Agile projects release more often than waterfall projects Availability of server hardware steadily growing (cloud) Previous release Stress and base costs no longer acceptable Groups involved must move closer together Affects development and operations in particular 34 17

First things first New buzzword necessary: ​​DevOps Portmanteau (case word) from Development and Operations First use of DevOpsDays 2009 in Belgium DevOps [] is a software development method that stresses communication, collaboration and integration between software developers and information technology (IT) professionals. (http://en.wikipedia.org/wiki/devops) 35 Houston, we have a problem (1) Two parties involved with opposing goals Development wants new features (change) Operations wants high and fast availability (stability) Competition since often no overarching view in companies Separation through wall of confusion Different goals and different tools We then throw Operations the next release over the fence (source: http://dev2ops.org/2010/02/what-is-devops/) 36 18

Houston, we have a problem (2) After a failed release, everyone plays The Blame Game Ops: Artifacts, scripts, config files, were faulty Dev: It worked for us, you did something wrong Ops: You have to look at what for yourself not true Dev: We don't even get to the prod machines (etc.) In the meantime nobody can work. The department doesn't care whether development or operations are to blame. Rollback of a partially implemented release is often impossible. Releases are not atomic, often in the area of ​​database changes As a consequence, somehow made executable Often dirty hacks without learning effect but with hidden errors 37 Déjà vu Wall of Confusion is not new Earlier separation between business and development solution was agile development DevOps should do the same for development and operations (source: http: // dev2ops. org / 2010/02 / what-is-devops /) 38 19

again Chaos Monkey Everything fails, all the time (Werner Vogels) Regularly terminates instances in a group of systems and indirectly tests whether the system continues to run anyway Strikes in controlled times to be ready for an emergency Transient environments Environments are as short-lived as possible (hours to days) No more sacred servers, with non-reproducible configuration Forced concept that everything must be automated 41 and again. Version everything The goal is to establish a single source of truth. A maximum of one checkout to get started for a new developer. Delivery pipeline Compare deployment pipeline from Continuous Delivery DevOps Dashboard Display of how changes affect the system in which stage and how Accessible to all groups involved Often anchored in the Continuous Integration Server 42 21

Question of the standpoint Continuous Delivery and DevOps are related to each other Similar or at least overlapping principles and key terms No mutual ignorance but rather the same goal Publications relate to the other term Flattening of release processes and organizational structures Desire are faster, cheaper and better releases 43 Structure Introduction Continuous Delivery DevOps summary 44 22

Summary Customer satisfaction requires delivery of software Classic release models do not fit modern development Continuous delivery is the logical consequence of continuous integration CI servers now also support deployment pipelines Delivery requires cross-team collaboration Mainly between development and operations Between development and operations there is a wall of confusion DevOps wants overcome this wall with methods and procedures 45 If you remember one thing In order for you to keep up with customer demand, you need to create a deployment pipeline. You need to get everything in version control. You need to automate the entire environment creation process. You need a deployment pipeline where you can create test and production environments, and then deploy code into them, entirely on demand. (The Phoenix Project) 46 23

Links Training: Version management with Git http://www.oio.de/git-schulung-versionsverwaltung-seminar-dvcstraining.htm Training: Version management with Subversion http://www.oio.de/subversion-svn-schulung.htm Training : Hudson Basics http://www.oio.de/schulung-hudson-seminar-continuous-integrationtraining-jenkins.htm Training: JIRA Technical Administration http://www.oio.de/seminar/methodik-prozess-management-softskills /seminar-training-atlassian-jira-schulung.htm 47 Links Article from the Java spectrum: Optimized testing (PDF) http://www.oio.de/public/softwaretest/optimiertestesten-gateways-gatekeeperkeymaster_js_03_11.pdf Advice on open source Tools: http://www.oio.de/beratung-consulting/open-sourcesoftware/tools/ 48 24

?? Ask??? Orientation Objects GmbH Weinheimer Str. 68 68309 Mannheim www.oio.de [email protected] ?? 49 Thank you for your attention! Orientation in Objects GmbH Weinheimer Str. 68 68309 Mannheim www.oio.de [email protected] 25