Security, Insights, and Results for your Drupal or WordPress Website - The Open Source for Business Solutions https://www.freelock.com/ en Fred Hutchinson Cancer Research Center eagle-i Integration https://www.freelock.com/partners/portfolio/fred-hutchinson-cancer-research-center-eagle-i-integration <span>Fred Hutchinson Cancer Research Center eagle-i Integration</span> <div class="field field--name-field-portfolio-extrashots field--type-image field--label-above"> <div class="field--label">Additional Screenshots</div> <div class="field--items"> <div class="field--item"> <img src="/sites/default/files/Selection_092.png" width="1015" height="619" alt="Fred Hutchinson" typeof="foaf:Image" class="img-responsive" /> </div> </div> </div> <span><a title="View user profile." href="/users/john-locke" lang="" about="/users/john-locke" typeof="schema:Person" property="schema:name" datatype="">John Locke</a></span> <span>Mon, 01/01/2018 - 00:00</span> Mon, 01 Jan 2018 08:00:00 +0000 John Locke 1025 at https://www.freelock.com https://www.freelock.com/partners/portfolio/fred-hutchinson-cancer-research-center-eagle-i-integration#comments University of Washington Center for Reinventing Public Education (CRPE) https://www.freelock.com/partners/portfolio/university-washington-center-reinventing-public-education-crpe <span>University of Washington Center for Reinventing Public Education (CRPE)</span> <div class="field field--name-field-portfolio-extrashots field--type-image field--label-above"> <div class="field--label">Additional Screenshots</div> <div class="field--items"> <div class="field--item"> <img src="/sites/default/files/crpe_full_homepage.png" width="1070" height="2937" alt="crpe_full_homepage" title="crpe_full_homepage" typeof="foaf:Image" class="img-responsive" /> </div> <div class="field--item"> <img src="/sites/default/files/portfolio_implementation_snapshot_too.png" width="1199" height="931" alt="portfolio_implementation_snapshot_tool" title="portfolio_implementation_snapshot_tool" typeof="foaf:Image" class="img-responsive" /> </div> </div> </div> <span><a title="View user profile." href="/users/don-dill" lang="" about="/users/don-dill" typeof="schema:Person" property="schema:name" datatype="">Don Dill</a></span> <span>Mon, 12/25/2017 - 12:56</span> Mon, 25 Dec 2017 20:56:12 +0000 Don Dill 1137 at https://www.freelock.com https://www.freelock.com/partners/portfolio/university-washington-center-reinventing-public-education-crpe#comments Seattle Humane Society https://www.freelock.com/partners/portfolio/seattle-humane-society <span>Seattle Humane Society</span> <span><a title="View user profile." href="/users/don-dill" lang="" about="/users/don-dill" typeof="schema:Person" property="schema:name" datatype="">Don Dill</a></span> <span>Wed, 12/20/2017 - 13:16</span> Wed, 20 Dec 2017 21:16:54 +0000 Don Dill 1677 at https://www.freelock.com https://www.freelock.com/partners/portfolio/seattle-humane-society#comments Middle East Policy Council https://www.freelock.com/partners/portfolio/middle-east-policy-council <span>Middle East Policy Council</span> <div class="field field--name-field-portfolio-extrashots field--type-image field--label-above"> <div class="field--label">Additional Screenshots</div> <div class="field--items"> <div class="field--item"> <img src="/sites/default/files/mepc_website_homepage.png" width="1599" height="2590" alt="MEPC homepage" typeof="foaf:Image" class="img-responsive" /> </div> </div> </div> <span><a title="View user profile." href="/users/don-dill" lang="" about="/users/don-dill" typeof="schema:Person" property="schema:name" datatype="">Don Dill</a></span> <span>Wed, 11/01/2017 - 12:48</span> Wed, 01 Nov 2017 19:48:15 +0000 Don Dill 1177 at https://www.freelock.com https://www.freelock.com/partners/portfolio/middle-east-policy-council#comments American College for Healthcare Sciences (ACHS) https://www.freelock.com/partners/portfolio/american-college-healthcare-sciences-achs <span>American College for Healthcare Sciences (ACHS)</span> <span><a title="View user profile." href="/users/don-dill" lang="" about="/users/don-dill" typeof="schema:Person" property="schema:name" datatype="">Don Dill</a></span> <span>Sun, 10/15/2017 - 15:26</span> Sun, 15 Oct 2017 22:26:21 +0000 Don Dill 1140 at https://www.freelock.com https://www.freelock.com/partners/portfolio/american-college-healthcare-sciences-achs#comments Peninsula College and Athletics Sites https://www.freelock.com/partners/portfolio/peninsula-college-and-athletics-sites <span>Peninsula College and Athletics Sites</span> <div class="field field--name-field-portfolio-extrashots field--type-image field--label-above"> <div class="field--label">Additional Screenshots</div> <div class="field--items"> <div class="field--item"> <img src="/sites/default/files/pencol_homepage.png" width="1594" height="931" alt="Peninsula College" typeof="foaf:Image" class="img-responsive" /> </div> <div class="field--item"> <img src="/sites/default/files/pencol_full_homepage.png" width="1049" height="1291" alt="Peninsula College" typeof="foaf:Image" class="img-responsive" /> </div> </div> </div> <span><a title="View user profile." href="/users/don-dill" lang="" about="/users/don-dill" typeof="schema:Person" property="schema:name" datatype="">Don Dill</a></span> <span>Mon, 10/02/2017 - 13:09</span> Mon, 02 Oct 2017 20:09:37 +0000 Don Dill 1138 at https://www.freelock.com https://www.freelock.com/partners/portfolio/peninsula-college-and-athletics-sites#comments Seattle Children's Alliance https://www.freelock.com/partners/portfolio/seattle-childrens-alliance <span>Seattle Children&#039;s Alliance</span> <div class="field field--name-field-portfolio-extrashots field--type-image field--label-above"> <div class="field--label">Additional Screenshots</div> <div class="field--items"> <div class="field--item"> <img src="/sites/default/files/childrensalliance.org_013_0.jpg" width="1890" height="2864" alt="Children&#039;s alliance homepage" typeof="foaf:Image" class="img-responsive" /> </div> </div> </div> <span><a title="View user profile." href="/users/don-dill" lang="" about="/users/don-dill" typeof="schema:Person" property="schema:name" datatype="">Don Dill</a></span> <span>Sun, 01/15/2017 - 18:06</span> Mon, 16 Jan 2017 02:06:47 +0000 Don Dill 1154 at https://www.freelock.com https://www.freelock.com/partners/portfolio/seattle-childrens-alliance#comments Queen City Yacht Club https://www.freelock.com/partners/portfolio/queen-city-yacht-club <span>Queen City Yacht Club</span> <span><a title="View user profile." href="/users/don-dill" lang="" about="/users/don-dill" typeof="schema:Person" property="schema:name" datatype="">Don Dill</a></span> <span>Sat, 07/16/2016 - 00:00</span> Sat, 16 Jul 2016 07:00:00 +0000 Don Dill 1121 at https://www.freelock.com https://www.freelock.com/partners/portfolio/queen-city-yacht-club#comments Aggregate fields in Drupal 8 views https://www.freelock.com/blog/john-locke/2020-04/aggregate-fields-drupal-8-views <span>Aggregate fields in Drupal 8 views</span> <span><a title="View user profile." href="/users/john-locke" lang="" about="/users/john-locke" typeof="schema:Person" property="schema:name" datatype="">John Locke</a></span> <span>Tue, 04/07/2020 - 12:51</span> <div class="field field--name-body field--type-text-with-summary field--label-hidden field--item"><p>Views module has long been the killer feature of Drupal, making it easy for a site builder or skilled administrator to essentially create complex SQL queries through a web interface, without knowing SQL. All kinds of things are possible through views - relationships, filters, sorting, access control, aggregation, argument handling, and more.</p> <p>What makes it even more powerful are views handlers. If there is some corner case you need to address, it's not that hard for a skilled developer to create a custom views handler to insert the SQL you need into the query, while exposing a nice configuration form for the site builder. There are tutorials and instructions all over the place to guide you in creating various kinds of field handlers, and you can also alter the query views generates through some "hook" code.</p> <p>Yet certain problems can be really hard to solve with views, and recently I came across two that were very similar:</p> <ol><li>A company holds multi-day seminars, and stores each day as a separate date range entry in a multi-value field, but wanted to list upcoming seminars with a single date range formatted with no duplication -- e.g. instead of "April 20, 2020 - April 20, 2020, April 21, 2020 - April 21, 2020, April 22, 2020 - April 22, 2020" they wanted it to just say "April 20 - 22, 2020". And not have the same class appear 3 times.</li> <li>For each event, they wanted a report showing how many people have registered to attend each upcoming seminar.</li> </ol><p>These seem like simple, obvious requests -- and yet they are nearly impossible to implement in views without some serious behind the scenes coding.</p> <h2>Deciding upon an approach</h2> <p>How do you solve a problem like this? There are many different ways to go about this. As an experienced Drupal developer, I came up with 4 different ways, and I'm sure there's others:</p> <ol><li>Create an aggregate view</li> <li>Create a custom views field handler</li> <li>Alter the views query</li> <li>Use a views subquery join handler and create an aggregate field.</li> </ol><p>Here's a brief run-down of these, with the one I chose and why.</p> <h3>Create an aggregate view</h3> <p>This is the obvious answer for creating reports: Check the box under "Advanced" to make the entire view an aggregate view. This changes each field to either be aggregated by some SQL aggregation function, or be used as a unique row.</p> <p>This approach is ok for summary reports, but it feels a bit like a sledgehammer, especially for the date range field -- it's a lot of extra work to set up an aggregate view, and it's extremely tricky to get right, especially when you go to filter out data to match exactly what you want. And, I have the tingly sense that I've tried this before and failed -- there are certain things you cannot do when you aggregate the entire view -- I can't necessarily put my finger on just what doesn't work in these cases, but I know it's not going to work in the end.</p> <h3>Create a custom views field handler</h3> <p>Drupal Console makes it really easy to generate a plugin for Drupal 8, and views handlers are all plugins. It's actually really easy to make a custom views handler, especially if you can base it on an existing handler and just override whatever it is you need to make custom.</p> <p>This seemed at first to be the way to go. For years, taxonomy fields were multi-valued -- e.g. multiple terms can be associated with a single node -- and the taxonomy field handlers have options to select whether you want an entirely new row for each term, or to collapse all the terms into a single field, perhaps separated by a comma.</p> <p>You would think this would work well for the date field, and it can certainly give us a count of attendees to an event with a little magic sql under the hood.</p> <p>I whipped up a field handler for the date, grabbing the earliest start date, the latest end date, and doing some ugly but workable code to show what the client wanted. And hit the problem with this approach: We didn't need "just" a field, but also a filter and a sort.</p> <p>We only wanted to show future seminars, and we wanted them sorted by date. But as soon as I added a date filter or sort, because there were multiple values, each seminar ended up with multiple rows, one for each of the date range values. Crap.</p> <p>I could make a filter handler. And a sort handler. But to get them to all use the same query started to feel -- dirty. Lots of repeating code, lots of altering queries or checking on whether the other handlers were already instantiated, lots of stuff that just felt wrong.</p> <p>Especially when I was in the views_data structures trying to hook them up -- it seemed entirely too hard.</p> <h3>Alter the views query</h3> <p>I probably spent upwards of 12 hours scouring the Internet, and the Drupal codebase for help or example code that might get me there. This seemed to be a problem that people just rolled their sleeves up and hacked their way through -- usually using a views_query_alter.</p> <p>No doubt, this is a fast way to get the job done -- just hook into the generated query, and alter it as you see fit.</p> <p>The biggest problem is, years down the road when you need to change something, you'll look at the view and say WTF? How does this thing even work? You end up with a views UI that just plain lies to you -- it does not make it clear what is really happening.</p> <p>I'll use a views_query_alter in a pinch, to get the job done -- but if you're reaching for this, it's usually because something wasn't done right.</p> <h3>Use a subquery join</h3> <p>Scouring the codebase, I found a views join handler called "subquery". That's exactly what I want to do -- create a small aggregate subquery, and join the main query to that. This solves both my problems -- I can create essentially a few aggregate fields while keeping the main sql query non-aggregated.</p> <p>The problem is, I don't think this handler even works! The details are here: <a href="https://www.drupal.org/project/drupal/issues/3125146">Issue #3125146 on Drupal.org</a>. I could not find a single place in the Drupal code base, or on the Internet, where this plugin was used. And it does not make sense to me how the SQL it generates is useful.</p> <p>But the concept is extremely useful -- having a subquery join makes a lot of sense. The challenge was, how to create and use it.</p> <h2>Implementing an aggregate "dummy" views field</h2> <p>So how to do it? The plugin/handler itself was the easy part. What was extremely hard was to figure out how to hook it up.</p> <p>In Drupal 8, the views_data structure basically is a registry of all the views handlers, mapped to the data structures they can handle. It is populated through the use of old-style Drupal hooks -- to add items to the views_data structure, you implement a "hook_views_data()" function in your module, and to change existing items, you implement a "hook_views_data_alter()" function. While I found snippets here and there on what to add here, it wasn't until I found a series of posts by Oleksandr Trotsenko about <a href="https://medium.com/oleksandr-trotsenko/drupal-8-views-tutorial-for-developers-part-i-theory-412d44b64adb">Drupal Views for Developers</a> that I figured out how to hook this all up.</p> <p>Go read that series first to learn how to create handlers and the views data structure. But there's some huge missing pieces about this data structure that I hope to illuminate here. Because that's where the true power lies...</p> <h3>The subquery join handler</h3> <p>First, the join handler that should be in core:</p> <pre> <code class="language-php">&lt;?php namespace Drupal\rng\Plugin\views\join; use Drupal\Core\Plugin\ContainerFactoryPluginInterface; use Drupal\views\Plugin\views\join\JoinPluginBase; use Symfony\Component\DependencyInjection\ContainerInterface; /** * Join handler for relationships that join with a subquery as a table. * * For example: * * @code * LEFT JOIN (SELECT subquery_fields[], subquery_expressions[] * WHERE subquery_where GROUP BY subquery_groupby) table * ON base_table.left_field = table.field * @endcode * * Join definition: same as \Drupal\views\Plugin\views\join\JoinPluginBase, * plus: * - subquery_fields[] * - subquery_expressions[] * - subquery_where * - subquery_groupby * * See https://www.drupal.org/project/drupal/issues/3125146. * * @ingroup views_join_handlers * @ViewsJoin("rng_subquery") */ class Subquery extends JoinPluginBase implements ContainerFactoryPluginInterface { /** * @var \Drupal\Core\Database\Connection */ protected $database; /** * {@inheritDoc} */ public static function create(ContainerInterface $container, array $configuration, $plugin_id, $plugin_definition) { $instance = new static($configuration, $plugin_id, $plugin_definition); $instance-&gt;database = $container-&gt;get('database'); return $instance; } /** * Builds the SQL for the join this object represents. * * @param \Drupal\Core\Database\Query\SelectInterface $select_query * The select query object. * @param string $table * The base table to join. * @param \Drupal\views\Plugin\views\query\QueryPluginBase $view_query * The source views query. */ public function buildJoin($select_query, $table, $view_query) { $alias = $this-&gt;configuration['subquery_alias']; $subquery = $this-&gt;database-&gt;select($this-&gt;configuration['subquery_table'], $alias); if (!empty($this-&gt;configuration['subquery_fields'])) { foreach ($this-&gt;configuration['subquery_fields'] as $field_alias=&gt;$field) { $subquery-&gt;addField($alias, $field, $field_alias); } } if (!empty($this-&gt;configuration['subquery_expressions'])) { foreach ($this-&gt;configuration['subquery_expressions'] as $field_alias=&gt;$expression) { $subquery-&gt;addExpression($expression, $field_alias); } } if (!empty($this-&gt;configuration['subquery_groupby'])) { $subquery-&gt;groupBy($this-&gt;configuration['subquery_groupby']); } if (!empty($this-&gt;configuration['subquery_where'])) { foreach ($this-&gt;configuration['subquery_where'] as $condition) { $subquery-&gt;where($condition); } } $right_table = $subquery; $left_table = $view_query-&gt;getTableInfo($this-&gt;leftTable); $left_field = "$left_table[alias].$this-&gt;leftField"; // Add our join condition, using a subquery on the left instead of a field. $condition = "$left_field = $table[alias].$this-&gt;field"; $arguments = []; // Tack on the extra. // This is just copied verbatim from the parent class, which itself has a // bug: https://www.drupal.org/node/1118100. if (isset($this-&gt;extra)) { if (is_array($this-&gt;extra)) { $extras = []; foreach ($this-&gt;extra as $info) { // Figure out the table name. Remember, only use aliases provided // if at all possible. $join_table = ''; if (!array_key_exists('table', $info)) { $join_table = $table['alias'] . '.'; } elseif (isset($info['table'])) { $join_table = $info['table'] . '.'; } $placeholder = ':views_join_condition_' . $select_query-&gt;nextPlaceholder(); if (is_array($info['value'])) { $operator = !empty($info['operator']) ? $info['operator'] : 'IN'; // Transform from IN() notation to = notation if just one value. if (count($info['value']) == 1) { $info['value'] = array_shift($info['value']); $operator = $operator == 'NOT IN' ? '!=' : '='; } } else { $operator = !empty($info['operator']) ? $info['operator'] : '='; } $extras[] = "$join_table$info[field] $operator $placeholder"; $arguments[$placeholder] = $info['value']; } if ($extras) { if (count($extras) == 1) { $condition .= ' AND ' . array_shift($extras); } else { $condition .= ' AND (' . implode(' ' . $this-&gt;extraOperator . ' ', $extras) . ')'; } } } elseif ($this-&gt;extra &amp;&amp; is_string($this-&gt;extra)) { $condition .= " AND ($this-&gt;extra)"; } } $select_query-&gt;addJoin($this-&gt;type, $right_table, $table['alias'], $condition, $arguments); } } </code></pre> <p>This is basically what the subquery join handler in core should look like.</p> <p>The "standard" join handler does have a "table formula" configuration that looks like is meant to provide the functionality we're trying to add here. That works, if you create a join handler programmatically and assign a DB Select query object to the "table formula" configuration -- but that's not possible to do from a views_data structure, because query objects aren't something you can serialize into configuration. And at the very bottom of the buildJoin method, the $select_query-&gt;addJoin method's second parameter is treated as a table name if it's a string -- and only added as a subquery if it's a Select object.</p> <p>So the only way to join a subquery is to build it inside this buildJoin() method, which means we need to pass everything needed to create that subquery in as configuration strings.</p> <h3>Views Data Structure</h3> <p>Here's the meat of the entire thing: Views Data. With a join handler like above, you can create all kinds of aggregate fields, without having to create a handler -- you can just let the subquery handle the aggregation and then use existing handlers for the field, filter, and sort.</p> <p>This ends up being much more elegant, and less confusing, than any of the other alternatives. However this structure is really lacking documentation, so that's what I hope to illuminate here. Read the comments inline...</p> <pre> <code class="language-php">/** * Implements hook_views_data(). */ function mymodule_views_data() { // The top level key is usually the entity type, or a database table, but can actually be anything. // Inside that top level key, you need to specify a literal 'table' element, and any number of "fields" $data['custom_registrants']['table'] = [ // The crucial thing to add is the 'join' key 'join' =&gt; [ // Under the join, the top level key is the "base table" -- e.g. what table is the left side of our join. // THIS IS IMPORTANT! Make sure this is the base table for your entity type -- if this base table is in // the view, then any fields defined as siblings to the 'table' key will be available in the view. 'commerce_product_field_data' =&gt; [ // IMPORTANT! 'join_id' must match the Plugin ID defined by the join plugin -- e.g. in the @ViewsJoin annotation. 'join_id' =&gt; 'rng_subquery', // The following are configurations available inside the handler. // subquery_alias is used inside the subquery for the main table. 'subquery_alias' =&gt; 'mymodule_summary', 'subquery_table' =&gt; 'registrant', // the key is the alias for the field, the value is the database column name. 'subquery_fields' =&gt; [ 'target_id' =&gt; 'event__target_id', ], // Expressions can use aggregate functions. The key is the alias for the expression. 'subquery_expressions' =&gt; [ 'num_registrants' =&gt; 'count(id)', ], 'subquery_where' =&gt; [ "event__target_type = 'commerce_product'", ], 'subquery_groupby' =&gt; 'target_id', // This is the id field of the base table (the key for the parent array). 'left_field' =&gt; 'product_id', // This field should be an alias of a field inside the subquery. 'field' =&gt; 'target_id', // This is the right side of the join -- will be the alias for the entire subquery, and must match // the root key of this array structure. 'table' =&gt; 'custom_registrants', ], ], ]; // This is a sibling of the "table" -- num_registrants is considered a "dummy" field on the "custom_registrants" // table -- which itself is the alias of the subquery. $data['custom_registrants']['num_registrants'] = [ 'title' =&gt; t('Attendee Count'), 'help' =&gt; t('Count of attendees registered for an event'), 'group' =&gt; t('Product'), 'field' =&gt; [ 'title' =&gt; t('Attendee count'), 'help' =&gt; t('Count of current attendees'), // What type of entity is this field available on 'entity_type' =&gt; 'commerce_product', // This is one of the field aliases inside the subquery 'field_name' =&gt; 'num_registrants', // This is the field handler plugin to use 'id' =&gt; 'standard', ], 'filter' =&gt; [ 'id' =&gt; 'numeric', 'title' =&gt; t('Num Registrants'), 'help' =&gt; t('Filter based on the count of attendees'), 'entity_type' =&gt; 'commerce_product', 'field_name' =&gt; 'num_registrants', ], 'sort' =&gt; [ 'id' =&gt; 'standard', 'title' =&gt; t('Num Registrants'), 'help' =&gt; t('Sort based on count of attendees'), 'real field' =&gt; 'num_registrants', ], ]; return $data; }</code></pre> <p>Now, that may seem like a lot of boilerplate. But the solution is elegant -- 6 lines for an array that makes it so this field can be added to the available columns to sort is all it takes! Now on a report that uses it, you can click the column header to sort by the most attended course to the least attended (or vice versa) without needing a custom handler. And the filter was just as easy!</p> <p> </p> </div> <div class="field field--name-taxonomy-vocabulary-5 field--type-entity-reference field--label-hidden field--items"> <div class="field--item"><a href="/tag/drupal-8" hreflang="en">Drupal 8</a></div> <div class="field--item"><a href="/tag/drupal-planet" hreflang="en">Drupal Planet</a></div> <div class="field--item"><a href="/tag/module-development" hreflang="en">Module Development</a></div> <div class="field--item"><a href="/tag/technical" hreflang="en">Technical</a></div> <div class="field--item"><a href="/tag/views" hreflang="en">Views</a></div> </div> <section> <article role="article" data-comment-user-id="0" id="comment-22646" class="comment js-comment by-anonymous clearfix"> <span class="hidden" data-comment-timestamp="1586629749"></span> <footer class="comment__meta"> <div class="col-sm-2 col-xs-4 comment-user-picture"> </div> </footer> <div class="comment-body col-sm-10 col-xs-8"> <div class="comment-arrow"></div> <div class="comment__content"> <div class="pull-left comment__author"><span lang="" typeof="schema:Person" property="schema:name" datatype="">darkdim</span></div> <div class="pull-right"><drupal-render-placeholder callback="comment.lazy_builders:renderLinks" arguments="0=22646&amp;1=default&amp;2=en&amp;3=" token="D1lw0vZ_lspSK7lL-T6WxdvOUouEpZaQdu57axvu2LE"></drupal-render-placeholder></div> <div class="field field--name-comment-body field--type-text-long field--label-hidden field--item"><p>Good job, Thank you!</p> </div> <div class="comment__time pull-right">08 Apr, 2020</div> </div> </div> </article> <article role="article" data-comment-user-id="5" id="comment-22663" class="comment js-comment by-node-author clearfix"> <span class="hidden" data-comment-timestamp="1586629815"></span> <footer class="comment__meta"> <div class="col-sm-2 col-xs-4 comment-user-picture"> </div> </footer> <div class="comment-body col-sm-10 col-xs-8"> <div class="comment-arrow"></div> <div class="comment__content"> <div class="pull-left comment__author"><a title="View user profile." href="/users/john-locke" lang="" about="/users/john-locke" typeof="schema:Person" property="schema:name" datatype="">John Locke</a></div> <div class="pull-right"><drupal-render-placeholder callback="comment.lazy_builders:renderLinks" arguments="0=22663&amp;1=default&amp;2=en&amp;3=" token="0Mlx_BChyKrx0YxzqrbuyhOsrk7OMVDZFPgAL51tU3Y"></drupal-render-placeholder></div> <div class="field field--name-comment-body field--type-text-long field--label-hidden field--item"><p>Update: code for this is now posted in the rng-2.x-dev branch. Not sure whether it will stay there, or get refactored into a different module.</p> </div> <div class="comment__time pull-right">11 Apr, 2020</div> </div> </div> </article> <h2>Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=2135&amp;2=comment_node_blog&amp;3=comment_node_blog" token="plWwx5-ySrYbSqEeyUDxUP3rqpOkfQrywPgS_AFXVLY"></drupal-render-placeholder> </section> Tue, 07 Apr 2020 19:51:51 +0000 John Locke 2135 at https://www.freelock.com Technical Debt and CMS maintenance https://www.freelock.com/blog/john-locke/2020-04/technical-debt-and-cms-maintenance <span>Technical Debt and CMS maintenance</span> <span><a title="View user profile." href="/users/john-locke" lang="" about="/users/john-locke" typeof="schema:Person" property="schema:name" datatype="">John Locke</a></span> <span>Mon, 04/06/2020 - 18:04</span> <div class="field field--name-body field--type-text-with-summary field--label-hidden field--item"><p>As we onboard a slew of new clients due to our <a data-entity-substitution="canonical" data-entity-type="node" data-entity-uuid="5cd9d344-1f5a-460a-8434-6420dc2db1d6" href="/blog/john-locke/2020-03/news-fuseiq-and-freelock-joining-forces" title="News: FuseIQ and Freelock joining forces!">joining forces with FuseIQ</a>, I wanted to take a moment to explain our stance on maintenance, particularly around applying non-security updates for Drupal and WordPress.</p> <p>Many people have a tentative approach to applying updates. "If it ain't broke, don't fix it!" is a saying we've all heard for generations, and sometimes it's hard to see changes as anything more than a risk you take that might potentially break things. But that's almost like saying "If I can't see it, it can't hurt me" -- in times of pandemic, does anyone really believe that?</p> <p>In security circles, there's a saying, "all bugs are security issues." The point being, anything the software does incorrectly bears some level of cost or risk to some set of users. Most people understand by now that if you don't fix a critical security issue, on the Internet you're likely to get found and hacked. But really, what is the cost of not applying a minor, non-security related issue?</p> <h2>Technical Debt</h2> <p>The concept of "technical debt" is that it is the sum total of all the stuff currently broken or outdated in your systems. It's usually related to what your ideal system might be, if you could have that today. If you do a great job building a site that does everything you need it to do, it may have no technical debt on the day of launch (quite unlikely, but at least possible), but this does not last. Why not?</p> <ul><li>Updates to the CMS software</li> <li>Updates to plugins or modules</li> <li>Updates to the software language</li> <li>Updates to the underlying server packages</li> <li>Updates to the underlying operating system</li> <li>Updates to web browsers</li> <li>New development toolkits that do more</li> <li>New devices that are substantially different than those that existed when you launched</li> <li>New or changing business requirements</li> <li>New partnerships</li> <li>New customer preferences</li> <li>New standards for search</li> <li>New ways to reach customers</li> <li>A pandemic changes your whole way of doing business</li> <li>Your entire business model has to change to keep up</li> </ul><p>... something in that list is likely changing every day. Hopefully mostly in small ways, but as we've all seen, sometimes in drastic ways. Regardless, if you are not keeping your site up to date, you are accruing "technical debt" that will need to get paid sooner or later, or else your site becomes less effective.</p> <p>And that is the true cost of doing nothing -- the opportunity cost of losing customers you might not have lost if you fully supported the newest device, or were able to communicate effectively to your customers how you can still deliver value to them when their entire world has changed.</p> <h2>The cost of "Security updates only"</h2> <p>Some sites are more complex than others, and some are more brittle than others, more likely to break in unexpected ways if anything changes. This is why many people suggest only applying security updates, and ignoring updates not marked as security-related.</p> <p>But there are some serious downsides to this approach:</p> <ol><li>Minor, incremental updates individually are far less risky than major updates -- going from one version to the next is far less likely than skipping 8 versions to apply a critical security update. Which means if there is a critical security update, the site is much more likely to break when you apply it, compared to if you had applied all the interim updates along the way.</li> <li>Environment changes sometimes get forced on you by hosting companies -- and many non-security updates fix issues with new versions. Not applying "regular" updates means more things broken on your site if this happens, and often more developer time to fix.</li> <li>It's far easier to automate updating everything, compared to just updating one security fix at a time.</li> <li>Most non-security releases fix bugs, or add new features, which might benefit your users -- making your site operate better or give you new abilities that help you stay relevant.</li> <li>If you're not fully up-to-date, and it takes longer to apply and test a security update, your site might be vulnerable to attack for a longer window. We've seen vulnerabilities where the time it took to get exploited was less than 2 days after a vulnerability was disclosed -- if it takes a week to update your site, you might have to pay the consequences of having a hacked site.</li> </ol><p>So in short, if your site is not kept fully up to date, it accrues a lot more technical debt. It becomes more expensive every time you apply an update, it carries more risk, and your users don't benefit from any of the improvements that might come with other regular updates.</p> <p>"But," you might ask, "isn't constant updating going to take a lot more of my time, cause more frequent breakages, and cost me more?"</p> <h2>How to keep Technical Debt under control</h2> <p>You're always going to have some level of technical debt. Your website is never going to be perfect. But it can be plenty good enough to be a huge value to your business or organization -- you just need to care for it in similar ways as you would care for any other property.</p> <p>If you think about a physical store, it's pretty clear there are regular maintenance needs. Once the store is built, the fixtures installed, inventory purchased and the shelves stocked, you still have constant things that need to happen:</p> <ul><li>Daily janitorial service</li> <li>Fix any broken windows</li> <li>Manage and check security systems</li> <li>Deep-clean carpets occasionally</li> <li>Fix holes in the roof</li> <li>Redesign the storefront</li> <li>Train the staff</li> <li>Come up with new merchandising fixtures to highlight specials</li> </ul><p>... the point is, successful retailers are constantly doing stuff to their stores, and constantly working with staff to improve sales. Your website, whether or not you do e-commerce, is exactly the same in this regard.</p> <p>What would happen to your store if you did not have janitorial services? If you left holes in the windows, or the roof? If you did not do a fresh paint coat now and then, or change up the storefront? If you did not train your staff?</p> <p>The point is, if your website is valuable to your business, you should stay on top of maintenance, and be constantly experimenting to see what works and how to improve sales. If you're not doing this, you will be falling behind your competitors who do.</p> <h2>How Freelock manages updates</h2> <p>As the store owner, you should not be doing janitorial work. You can outsource that easily. (Sure you can joke about being the head janitor, and pick up a broom now and then if you'd like, but it's not the job that only you can do).</p> <p>Freelock can do all the maintenance work for you, for far less cost than you doing it yourself, or even having a staff member do it. We have automated a large part of the process -- particularly automatically running two kinds of tests with every release, backing up sites before and after every release, and checking every site every night for changes.</p> <p>It's far easier, and less costly, to apply all updates any time we touch a site, than to limit updates to just security releases. And with tests in place to catch things that break, we can do this with high confidence that the update does not cause major issues.</p> <p>It is fairly common for an update to cause a minor issue, however. And this is exactly where technical debt comes back in -- it's far easier to pay that upgrade cost one small issue at a time, as we go -- instead of ending up having to fix a dozen small issues that combine into one big showstopping issue, all at once, under pressure due to a known security risk. And by "cost", in this case it's the cost of demanding your attention and ours when we're both swamped with other demands.</p> <p>So... When we take over a site, there is a higher-than-usual cost to get you set up, brought completely up-to-date, and create the tests to cover your particular critical site needs. Once all of that setup is done, our maintenance cost tends to be lower than many other firms, thanks to our automation as well as our hands-on approach to providing fixes -- once we've hit an issue on one site, and resolved it, we usually can apply that fix immediately to any other site we manage that has the same issue.</p> <p>Our "<a data-entity-substitution="canonical" data-entity-type="commerce_product" data-entity-uuid="1aeb4221-9364-4539-8236-10820ee61c98" href="/product/protection-plan" title="Protection Plan">protection plan</a>" is the base maintenance we provide, for either <a data-entity-substitution="canonical" data-entity-type="commerce_product" data-entity-uuid="e7cd0cc6-9a7f-4348-aba5-5bef1ced9165" href="/product/drupal-protection-plan" title="Drupal Protection Plan">Drupal</a> or <a data-entity-substitution="canonical" data-entity-type="commerce_product" data-entity-uuid="2070c57d-31de-4fe5-8ff8-962c542fb23f" href="/product/wordpress-protection-plan" title="WordPress Protection Plan">WordPress</a>. With these plans, we generally apply all updates to all our managed sites, on a monthly basis. We monitor security lists, and if there's a critical security vulnerability we judge to affect your site, we apply that within 1 business day, and security vulnerabilities we deem not a risk for your site we usually apply within 1 week.</p> <p>Feel free to <a href="/contact">reach out</a>, or comment below, if you have any questions or feedback!</p> <p>Cheers,</p> <p>John</p> </div> <div class="field field--name-taxonomy-vocabulary-5 field--type-entity-reference field--label-hidden field--items"> <div class="field--item"><a href="/tag/drupal" hreflang="en">Drupal</a></div> <div class="field--item"><a href="/tag/drupal-planet" hreflang="en">Drupal Planet</a></div> <div class="field--item"><a href="/tag/maintenance" hreflang="en">maintenance</a></div> <div class="field--item"><a href="/taxonomy/term/598" hreflang="en">Technical Debt</a></div> <div class="field--item"><a href="/tag/wordpress-0" hreflang="en">WordPress</a></div> </div> <section> <h2>Add new comment</h2> <drupal-render-placeholder callback="comment.lazy_builders:renderForm" arguments="0=node&amp;1=2134&amp;2=comment_node_blog&amp;3=comment_node_blog" token="fGMa4H4684QQym9uLoh_VzwmE32sKeMNIfqzsCl2aH0"></drupal-render-placeholder> </section> Tue, 07 Apr 2020 01:04:25 +0000 John Locke 2134 at https://www.freelock.com