Ignoring outliers in aggregate function

This is another aide-memoire for myself really.  I want to calculate the average load times per page for an application from timings stored in the database, and see which pages need attention. However, the stats can be skewed by the odd exceptional load that takes much longer than a typical load for reasons that are probably irrelevant to me.

Here is a fictitious example:

create table timings (id int, timing number);

insert into timings
select rownum, case when rownum=50 then 1000 else 1 end
from dual
connect by rownum <= 100;

This example has 99 timings of 1 second plus an oddity of 1000 seconds.

A simple average gives a skewed picture:

SQL> select avg(timing) from timings;

AVG(TIMING)
-----------
      10.99

It suggests that users are waiting 11 seconds on average for a page to load, when in fact it is usually 1 second.

The analytic function NTILE(n) can solve this.  This divides the set of results into n "buckets" and then tells us which bucket a particular value falls into.  If we do that with a suitable number of buckets, say 10, we will be able to exlude the highest 10% and lowest 10% of the values:

SQL> select avg(timing) from 
  2  (select timing, ntile(10) over(order by timing) bucket
  3   from timings)
  4  where bucket between 2 and 9;

AVG(TIMING)
-----------
          1

Why use CASE when NVL will do?

I've found that many developers are reluctant to use "new" features like CASE expressions and ANSI joins. (By new I mean: this millennium.)

But now they have started to and they get carried away.  I have seen this several times recently:

    CASE WHEN column1 IS NOT NULL THEN column1 ELSE column2 END

Before they learned to use CASE I'm sure they would have written the much simpler:

    NVL (column1, column2)

Now I now that NVL is Oracle-specific and CASE is portable, but (a) we aren't ever going to be porting our millions of lines of PL/SQL, and (b) I can guarantee they didn't do it for that reason. They have got a new hammer and now everything looks like a nail.

Handy pre-defined Oracle collections

Note to self:

1. SYS.DBMS_DEBUG_VC2COLL is a handy pre-defined TABLE OF VARCHAR2(1000)
2. SYS.KU$_VCNT is TABLE OF VARCHAR2(4000)

Both are granted to public.

Thanks to Eddie Awad's blog for these.

Hiding APEX report pagination when trivial

The users are quite happy with pagination like this:

However, they don't like it when the report returns less than a pageful of rows and they see this:

(Fussy, I know).

This is one way to do it.  First, ensure that the pagination area itself is identifiable.  I put a div around it with a class of "pagination":

Then add some Javascript to the "Execute when page loads" attribute of the page:

$('div.pagination').each(function() {
    if ($(this).find('td.pagination a').length == 0
       && $(this).find('div.msg').length == 0
       ) {
    $(this).hide();
    }
})

It looks for pagination areas that contain no links and no “reset pagination” error message, and hides them.

The Javascript could go into the page templates to fix the issue across all pages.

APEX boilerplate translation

APEX provides a mechanism for translating applications into other languages:

Applications can be translated from a primary language into other languages. Each translation results in the creation of a new translated application. Each translation requires a mapping which identifies the target language as well as the translated application ID. Translated applications cannot be edited directly in the Application Builder.
Once the translation mappings are established the translatable text within the application is seeded into a translation repository. This repository can then be exported to an XLIFF for translation.
Once the XLIFF file is populated with the translations, one file per language, the XLIFF file is uploaded back into the translation repository. The final step is to publish each translated application from the translation repository.
A translated application will require synchronization when the primary application has been modified since the translated version was last published. Even modifications to application logic will require synchronization. To synchronize, seed and publish the translated application.

I have never used this method, but I have worked on a number of APEX applications that can be translated into other languages by other means.  Essentially this is a matter of "soft-coding" all boilerplate such as region titles and item labels - holding them as data in tables and selecting the appropriate values at run time.

For item labels there is a neat solution involving APEX shortcuts.  First we define a table to hold the item labels in all the languages we support something like this:

APP_ID	ITEM_NAME	LANGUAGE	LABEL_TEXT
101	P1_EMPNO	ENG	Employee identifier
101	P1_EMPNO	ES	Identificación del empleado
101	P1_EMPNO	FR	Identifiant des employés

Now we need a function to get the label text for an item in the current language, something like:

function label_text (p_app_id number,
                     p_item_name varchar2,
                     p_language varchar2)

Then we can create an APEX label template with "Before label" HTML like this:

<label for="#CURRENT_ITEM_NAME#">"LABEL_TEXT"

"LABEL_TEXT" is a reference to an APEX shortcut which we can now define using PL/SQL function body:

return translate_pkg.label_text(:APP_ALIAS,'#CURRENT_ITEM_NAME#',:AI_LANGUAGE)

(AI_LANGUAGE is an application item defining the user's preferred language.) All we now need to do is use the new label template on all our page items and leave the item's label attribute blank.

It would be nice if we could do something similar for region titles, but unfortunately APEX shortcuts are not currently supported in region titles. Instead we have to make do with creating a hidden item e.g. P1_EMP_REGION_TITLE that we set to the desired title, and then set the region title to &P1_EMP_REGION_TITLE. That still leaves the matter of translating error messages and of course the actual data, but I won't go into that in this post.

HGV Levy

The UK government has introduced a new service for foreign lorry drivers to pay a levy to use UK roads here:

https://www.hgvlevy.service.gov.uk/

It was built by my current employer, Northgate Information Solutions. Guess what technology it runs on?
We had a lot of interesting challenges when building this:

• Compliance with UK Government styling and standards
• Responsive design to work on desktops, tablets and mobiles
• Accessibility
• Usable with Javascript disabled
• Translated into 5 European languages

You can read more about this project here on the Northgate website.

It's a drag...

It really used to be a "drag" putting together a set list for my band using my Oracle database of songs we play: the easiest way was to download all the songs to an Excel spreadsheet, manipulate them there, then re-import back into the database.  I tried various techniques within APEX but none was easier than that - until now.  I have just discovered jQuery UI's Sortable interaction, and in a couple of hour was able to build exactly what I'd always needed:

(That is is one of our recent set lists: I never claimed we were cool - we're called The Love Handles after all.)

The full list of songs (minus any already selected) appears in the first list.  Then there are 4 more lists corresponding to sets (we usually have 2 sets, with a 3rd to hold "spares" in case we under-run or get asked to play on.  Occasionally we play 3 sets.) I can now:

• Add a song to a set by dragging it from the Songs list to the desired Set list
• Remove a song from a set by dragging it from the Set list back to the Songs list
• Move songs from one set to another by drag and drop
• Move a song up and down within a set by drag and drop

It really was incredibly simple to achieve.  First I needed to include the relevant jQuery library into my page:

#IMAGE_PREFIX#libraries/jquery-ui/1.8.22/ui/minified/jquery.ui.sortable.min.js

(This is shipped with APEX but not included in applications by default.)

Then I constructed the HTML for each list as follows:

<ul class="connectedSortable">
<li class="set1"><input type="hidden" name="f01" value="123" />Land of 1000 Dances</li>
...
</ul>
<input type="hidden" name="f01" value="0" />

Explanation:

• Each of the 5 lists has the same class "connectedSortable".  This is what allows us to drag and drop between them.
• Each list item consists of a hidden item containing the song's ID in the database, and the song title to be displayed.  All the hidden items have name="f01" so that they will all appear in an APEX array g_f01 when the page is submitted. (The class "set1" is only there so that I can colour each list differently.)
• After each list I put another hidden item also named "f01" with a value of 0.  This acts as a separator between the lists when processing the g_f01 array - if the value is 0 I know that I have reached the end of a list and am starting the next.
• Originally, I built each list as a report region, with a bespoke region and report template.  But after a while I realised it was simpler to use PL/SQL regions to dynamically render the HTML.

Now for the Javascript that makes it all work:

  $(function() {
    $( ".connectedSortable" ).sortable({
      connectWith: ".connectedSortable"
    }).disableSelection();
  });

That's it - just that.

Finally I just had to write some PL/SQL to process the array when the page is submitted:

declare
   l_set_no integer := 0;
begin
   apex_collection.create_or_truncate_collection ('SETLIST_EDITOR');
   for i in 1..apex_application.g_f01.count
   loop
      if apex_application.g_f01(i) = 0 then
         -- End of set
         l_set_no := l_set_no + 1;
      else
         -- Song: add to current set
         apex_collection.add_member
            ( 'SETLIST_EDITOR'
            , l_set_no
            , apex_application.g_f01(i)
            );
      end if;
   end loop;
end;

I'm using a collection here, which I populated on initial page load from the database. I could have worked on the set list table directly, but this allows me to submit the page if needed for other reasons without saving or losing the changes.

You can see the final result on this apex.oracle.com demo.

I think jQuery is now my second-favourite developer tool (after APEX of course).