Showing posts with label oracle. Show all posts
Showing posts with label oracle. Show all posts

Tuesday, March 17, 2020

Oracle APEX Global Notification

In the APEX application properties there is a property called Global Notification where you can enter some text that will be displayed on every page - the help text says:
You can use a global notification to communicate system status. If your page template contains a #GLOBAL_NOTIFICATION# substitution string then the text entered here displays on each page. For example, suppose you entered the message "Team picnic this Friday" in this attribute. Assuming your page templates support the global notification substitution string then this message would display on each page.
To create a global notification:
1. Include the #GLOBAL_NOTIFICATION# substitution string in your page template.
2. Navigate to the Edit Application page and enter a message in the Global Notifications attribute.
3. Click Apply Changes.
So let's see what we get if we put our message here:
It works, but it isn't exactly eye-catching is it? I was hoping the page template might have some mark-up around this to liven it up, but it doesn't. I could change the page template, but I'd rather not do that as I'd prefer to use standard Universal Theme page templates that can be refreshed after upgrades. Or I could use HTML in the global notification text:
<div class="u-info padding-md">
  <span class="fa fa-info-circle"></span>
  The system will be down for essential maintenance over the weekend.
</div>
That looks like this:
That is a lot better, I think. And obviously I can vary the colour and icon according to the type of message. But it still has some drawbacks for me:
  • I need to remember that HTML mark-up each time I want to set up a global notification, or remove one.
  • I need to amend the application to do it, rather than change some data outside the application.
So I quickly devised an alternative global notification system using a table and a report region with a bespoke report template. Here is the table:
The ID is just a surrogate key. TEXT contains the plain text of the message, e.g. "The system will be down for essential maintenance over the weekend." FROM_DATETIME and TO_DATETIME allow set-up of a message that will show for a defined period only (these are optional in case I want one shown indefinitely.) Finally, STATE is a string such as 'danger', 'info', 'success', 'warning' to determine the appropriate styling and icon to use. Here is the report SQL:

select id, text
, case state
       when 'info' then 'fa-info-circle'
       when 'danger' then 'fa-hand-stop-o'
       when 'warning' then 'fa-warning'
       when 'success' then 'fa-check-circle'
       end as iconclass
, 'u-' || state as divclass       
from notices
where sysdate between nvl(from_datetime,sysdate) and nvl(to_datetime,sysdate)
and id not in (select column_value from apex_string.split(:APP_NOTICES_CLOSED))
order by case state when 'danger' then 1 when 'warning' then 2 end, state

The report SQL derives the icon type and the class to style the message based on the state. An enhancement would be to put these in a table of notification state attributes and look them up. The report region is defined on page 0 so that notifications appear on all pages, in the Before Content Body position. The region template is Blank with Attributes. I have created a bespoke report (row) template - I could have used the Universal Theme's "Alert" report template, but it isn't quite what I want. The template HTML is:

<div class="#DIVCLASS# padding-md">
  <span class="fa #ICONCLASS#"></span>
  #TEXT#
  <a href="#" id="closeNotice-#ID#" class="closeNotice" title="Hide this notice">
    <span class="fa fa-times-circle-o u-pullRight #DIVCLASS#"></span>
  </a>
</div>

OK, let's set up a few notices and see what we get...
Nice(ish). But the users might get fed up with all those messages clogging up every page. What they need is a way to dismiss them. We've already done some of the work - note the X icon at the right-hand end of each message, and this line in the report SQL:
and id not in (select column_value from apex_string.split(:APP_NOTICES_CLOSED))
We need to create an application item called APP_NOTICES_CLOSED. This will be a list of the notice IDs that the user has dismissed. Then we need the logic to maintain this item - a dynamic action defined on page 0 as follows:
  • Event: Click on Javascript Selector ".closeNotice"
  • Action: Execute Javascript code:
  • $s ('P0_CLOSE_NOTICE_ID', 
        $(this.triggeringElement).attr('id').substr(12));
    (So we also need a hidden page 0 item called P0_CLOSE_NOTICE_ID)
  • Action: Execute PL/SQL code:
  • declare
       l_count integer;
       l_tab apex_t_varchar2;
    begin   
       select count(*)
       into l_count
       from apex_string.split(:APP_NOTICES_CLOSED)
       where column_value = :P0_CLOSE_NOTICE_ID;
       
       if l_count = 0 then
          l_tab := apex_string.split(:APP_NOTICES_CLOSED);
          l_tab.extend;
          l_tab(l_tab.count) := :P0_CLOSE_NOTICE_ID;
          :APP_NOTICES_CLOSED := apex_string.join(l_tab);
       end if;   
    end;   
  • Action: Hide - Javascript expression:
    $(this.triggeringElement).closest('div');
Now, when the user clicks on the X on a notification message, we hide it and add its ID to a list of notifications that we will no longer show in this session. Next time the user logs on, all current messages will be displayed again. An obvious enhancement would be a way to dismiss messages forever once acknowledged.

So there it is - a fairly quick and dirty global notification mechanism with scope for various enhancements.

Wednesday, July 15, 2015

Another new APEX-based public website goes live

Another APEX public website I worked on with Northgate Public Services has just gone live:

https://londontribunals.org.uk/

This is a website to handle appeals against parking fines and other traffic/environmental fines issues by London local authorities.

It is built on APEX 4.2 using a bespoke theme that uses the Bootstrap framework.  A responsive design has been used so that the site works as well on a mobile phone as on a desktop.

Rumours that appeals against any parking tickets with my car's registration number on them are automatically approved by the system are completely unfounded.

Saturday, August 30, 2014

Handy pre-defined Oracle collections

Note to self:

  1. SYS.DBMS_DEBUG_VC2COLL is a handy pre-defined TABLE OF VARCHAR2(1000)
  2. SYS.KU$_VCNT is TABLE OF VARCHAR2(4000)

Both are granted to public.

Thanks to Eddie Awad's blog for these.

Wednesday, September 18, 2013

SQL Developer: add a "child tables" tab to table definition

I always like to extend SQL Developer's table definition by adding a tab that shows the "child tables" for each table - i.e. the tables that have a foreign key to the table in context.

I have lost count of how many times I have started work at a new environment or on a new PC and had to set this up from scratch, so thought I'd document it here for next time!

First, create a file containing the following XML:

<items>
  <item type="editor" node="TableNode"  vertical="true">
      <title><![CDATA[Child Tables]]></title>
         <query>
             <sql><![CDATA[select cons.table_name, cons.constraint_name
                      from all_constraints cons
                      where cons.constraint_type = 'R'
                      and (cons.r_constraint_name, cons.r_owner) in
                                         (select pk.constraint_name, pk.owner
                                          from   all_constraints pk
                                          where owner = :OBJECT_OWNER 
                                          and table_name = :OBJECT_NAME
                                          and constraint_type in ('P','U'))]]>
             </sql>
         </query>
   </item>
</items>

Then, in SQL Developer go to Tools->Preferences... and open the Database node and select user Defined Extensions.

Click Add Row and on the new row set Type to EDITOR and Location to point to your XML file.

That's it.  You may just need to restart SQL Developer to make it work.

I am indebted to Sue Harper's blog for the detailed instructions on adding tabs to SQL Developer.

Another tab I like to add is one to show the errors for an invalid view definition:

<items>
 <item type="editor" node="ViewNode"  vertical="true">
    <title><![CDATA[ERRORS]]></title>
      <query>
         <sql><![CDATA[SELECT
     ATTRIBUTE, LINE
     || ':'
     ||POSITION "LINE:POSITION", TEXT
FROM
     All_Errors
WHERE
     type  = 'VIEW'
 AND owner = :OBJECT_OWNER
 AND name  = :OBJECT_NAME
ORDER BY
     SEQUENCE ASC
         ]]></sql>
      </query>
 </item>
</items>

(Update 2019: SQL Developer now includes an Errors tab for views already, so this is redundant.)

Friday, June 04, 2010

Shameless boasting

I hate to boast but...

StackOverflow has become one of my favourite forums for reading and sometimes answering Oracle-related questions (though it covers all programming topics in fact).

Today I am the first person ever to be awarded the Oracle badge for having earned 1000 upvotes for my answers to questions with the Oracle tag:


Of course, this may just mean I have too much time on my hands...

Sunday, February 15, 2009

UKOUG - Apex SIG

I was lucky enough to be able to attend the UK Oracle User Group's Application Express SIG on Friday - my first experience of any UKOUG event. The event was a "sell out", with all available places taken up well in advance. This in itself is great news, because it shows that APEX really is taking off, which means I can be fairly confident I haven't backed the wrong horse career-wise.

David Peake, the APEX Product Manager, gave the first two presentations:

  • The Latest and Greatest from Development

  • APEX @ Oracle


The first of these was a tantalising glimpse at what will be in 3.2 (coming soon) and 4.0 (coming later this year hopefully). These are described in the APEX Statement of Direction, but we were able to see some of them in a live demo. I am particularly looking forward to 4.0 for its improved tabular forms, and for Dynamic Actions - i.e. a "rich client" experience without having to write any Javascript.

Dimitri Gielis and John Scott from Apex Evangelists both gave interesting presentations, on Charting in Apex and Dispelling Myths about APEX respectively. Dimitri's piqued my interest to go back and look at charts again, and John's provided a lot of useful ammo for fighting against the usual "Apex is just a power user toy" kind of myths.

There was also a presentation from Matt Nolan and Vincent Migue from e-DBA called "Using Apex to Expose your Business to the Web" - and their website is a fine example of that.

After a short Q&A forum, we all went to the local pub, which was a great opportunity to get to know some of these people a little better.

All in all, a great day out (good grief, how sad do I sound?!) and I'd definitely like to attend the next one, should there be one.

Thursday, November 01, 2007

Apex Impact Analysis script (v2)

After receiving useful comments from Patrick Wolf on my previous post, here is an enhanced version of the script that also checks condition expressions. I'm not including condition_expression2 because I don't think that ever contains anything other than literal values (am I right?)



column obj_name format a50

undef search_text

accept search_text prompt "Enter search text: "

select application_id, page_id, 'Region' objtype, region_name obj_name, 'Source' usage_type
from apex_application_page_regions
where lower(region_source) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Item' obj_type, item_name obj_name, 'Source' usage_type
from apex_application_page_items
where lower(item_source) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Process' obj_type, process_name obj_name, 'Source' usage_type
from apex_application_page_proc
where lower(process_source) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Branch' obj_type, TO_CHAR(process_sequence) obj_name, 'Source' usage_type
from apex_application_page_branches
where lower(branch_action) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Region' obj_type, region_name obj_name, 'Condition' usage_type
from apex_application_page_regions
where lower(condition_expression1) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Item' obj_type, item_name obj_name, 'Condition' usage_type
from apex_application_page_items
where lower(condition_expression1) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Item' obj_type, item_name obj_name, 'Read Only' usage_type
from apex_application_page_items
where lower(read_only_condition_exp1) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Process' obj_type, process_name obj_name, 'Condition' usage_type
from apex_application_page_proc
where lower(condition_expression1) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Branch' obj_type, TO_CHAR(process_sequence) obj_name, 'Condition' usage_type
from apex_application_page_branches
where lower(condition_expression1) like '%&search_text.%'
ORDER BY 1,2,3,4
/

Wednesday, October 31, 2007

Apex: impact analysis script

Ever wondered where a particular package is being used by your Apex application(s)? This simpleSQL Plus script may be of use:



column obj_name format a50

undef search_text

accept search_text prompt "Enter search text: "

select application_id, page_id, 'Region' objtype, region_name obj_name
from apex_application_page_regions
where lower(region_source) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Item' obj_type, item_name obj_name
from apex_application_page_items
where lower(item_source) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Process' obj_type, process_name obj_name
from apex_application_page_proc
where lower(process_source) like '%&search_text.%'
UNION ALL
select application_id, page_id, 'Branch' obj_type, TO_CHAR(process_sequence) obj_name
from apex_application_page_branches
where lower(branch_action) like '%&search_text.%'
ORDER BY 1,2,3,4
/


For example:

Enter search text: my_package.my_fun

APPLICATION_ID PAGE_ID OBJTYPE OBJ_NAME
-------------- ---------- ------- ------------------------------------------
101 123 Process Call my function
101 127 Branch 30
102 128 Region My Report
102 129 Region My Query
103 21 Item P21_MY_ITEM


Feel free to tell me if (a) I have missed anything, or (b) I have re-invented the wheel and should be using some built-in Apex utility!

Wednesday, January 04, 2006

Why I Love HTMLDB

Over the last 3 years or so I have got to know HTMLDB well, and have become a big fan of it - as a development tool, not as a "power user toy". I have successfully built and deployed a number of HTMLDB applications. I find it appeals to me in a way that .Net and J2EE never have. Why is this? Here are some reasons:

  • all programming is in PL/SQL, my favourite programming language

  • all the "boring bits" like pagination of record sets are built-in

  • integration with the Oracle database is seamless

  • you don't need a big application server to support it

  • there is no question of reinventing the wheel by caching data, locking data, implementing business rules in the "middle tier" (there is NO "middle tier"!)


There are, I must admit, some shortcomings:

  • it was not designed with large applications and large development teams in mind, so code and build management is a challenge (but one that we are learning to overcome)

  • some of the functionality is not yet quite as robust as it could be (hopefully this will improve as the product matures)


However, HTMLDB doesn't seem to be to everyone's taste. Some hardened J2EE fanatics think it is a toy that is not good enough to be used by "programming professionals" - see for example this exchange I had on Oracle WTF.

I guess one of the key comments from the J2EE fan was this: "and as an ex Forms programmer NOTHING is a backward step from oracle forms!"

Now, I will admit that Oracle Forms is now getting rather old and tired - but in its heyday (around 1990-1995) it was a great product. What it did was integrate seamlessly with the database and do all the drudgery of paginating record sets etc. while you got on with implementing the business requirements. In fact, all the reasons I gave above for liking HTMLDB apply equally to Oracle Forms.

But for some years now, Forms has been looking like a "legacy system" - shoe-horned into the ubiquitous browser interface as WebForms but never looking quite at home there. And the alternatives I saw seemed to be a step backwards in terms of data management:

  • ASP/.Net: write lots of VB code and hand-crank things like record pagination

  • J2EE: write lots of Java code (which looks a LOT like C!), build your own classes to wrap around database objects etc. And cope with the "standards" that came and went like pop idols (JSF is it currently?)


Then I came across HTMLDB, promoted by Tom Kyte on his AskTom website. You could try it out for free, without even downloading anything, on Oracle's hosted HTMLDB development website. Of course, as with anything, there was a learning curve to get over, which probably took me a few months (elapsed: I was only playing with HTMLDB in my spare time.) But what was very soon apparent to me was that this could be the successor to Oracle Forms that I had been waiting for.

However, in their wisdom Oracle have decided to position HTMLDB as a "power user toy" to replace Excel and Access for very small databases - which it certainly can do. But this marketing undermines HTMLDB as a tool for "serious" application developers. It has been suggested to me that this may be because HTMLDB didn't come out of the Oracle Tools group, which of course has JDeveloper to promote.

Once I had built some small-scale applications with HTMLDB to replace aging Forms applications used only internally, the people I work for were sufficiently impressed to consider using it for something a lot bigger. I built a prototype which was received favourably, but we knew we ought to at least look at JDeveloper with ADF, since that is the approach Oracle recommend for moving away from Forms. So we got Oracle to send us someone to demonstrate JDeveloper to us. It all looked very slick, knocking up an "emp and dept" application in a few minutes - but nothing that couldn't have been done just as easily with HTMLDB. And when he tried to customise the application at our request, he quickly came unstuck. Soon afterwards, a colleague went to an Oracle "Developer Day" (or some such title) where JDeveloper was demoed agan. This time, the compilation process failed with a fatal error, and the demonstrator gave up and moved on to something else. What with these unimpressive demonstrations and the fact that our business logic already exists in the form of PL/SQL packages, it was very clear that HTMLDB was the only realistic option to replace the system within the timescale and budget allowed. (My impression is that JDeveloper might be really good in about 3 years time!)

But even if we were building a brand new system, my philosophy would be to build the business logic as PL/SQL packages and then build a light UI application on top of that: I really don't buy the whole J2EE way of doing things. It implies that the application is king, rather than the data. As Tom Kyte often says: applications come and go, but the data lasts "for ever".

So, for now at least, I am staking my future on HTMLDB rather than J2EE. Maybe I'll regret that one day, but I doubt it.

Monday, January 02, 2006

PL/SQL So Doku Solver

So Doku has taken over the UK, if not the world, in the last year. I am now addicted, but at first I couldn't see the point in these puzzle and prefered to write a program to solve them for me. What was the point in that? Absolutely none! But in case you ever feel the need to get a Su Doku puzzle solved without actualy doing it yourself, here is the code - think of it as a late Christmas present!

First, we need to create a table:

create table cells
( x number(1,0) not null
, y number(1,0) not null
, z varchar2(9)
, done varchar2(1) not null
, constraint cells_pk primary key (x, y)
)
/


Then a package:

create or replace package cell_pkg as
procedure solve(p_state in varchar2);
procedure print;
end;
/

create or replace package body cell_pkg as
procedure solve(p_state in varchar2)
is
v varchar2(1);
cnt integer;
processed integer;
it_failed exception;
begin
-- Set up a clean board
delete cells;
for r in 1..9 loop
for c in 1..9 loop
insert into cells (x,y,z,done) values (r,c,'123456789','N');
end loop;
end loop;
-- Apply initial state
for r in 1..9 loop
for c in 1..9 loop
v := substr(p_state,(r-1)*9+c,1);
if v between '1' and '9' then
update cells
set z = v
where x = r
and y = c;
end if;
end loop;
end loop;
-- Start processing
loop
processed := 0;
-- Process cells that are solved but not yet marked as "done":
for rec in (select * from cells where length(z)=1 and done='N')
loop
-- Remove that cell's value from the possible values for other cells
-- in same row, column or square
update cells
set z = replace(z,rec.z)
where ( x = rec.x
or y = rec.y
or ( floor((x-1)/3) = floor((rec.x-1)/3)
and floor((y-1)/3) = floor((rec.y-1)/3)
)
)
and (x <> rec.x or y <> rec.y); -- Exclude self!
-- Now mark this cell as done
update cells
set done = 'Y'
where x = rec.x and y = rec.y;
-- Note how many cells processed on this pass
processed := processed+1;
end loop;
-- Look for cells that are not solved, but where they are the only cell
-- in their row, column or square containing a given value
for i in 1..9 loop
for rec in (select * from cells c1
where length(z) > 1
and z like '%'||i||'%'
and ( not exists
(select null
from cells c2
where c2.x = c1.x -- Same row
and (c1.x <> c2.x or c1.y <> c2.y) -- Exclude self!
and c2.z like '%'||i||'%'
)
or not exists
(select null
from cells c2
where c2.y = c1.y -- Same column
and (c1.x <> c2.x or c1.y <> c2.y) -- Exclude self!
and c2.z like '%'||i||'%'
)
or not exists
(select null
from cells c2
where floor((c2.x-1)/3) = floor((c1.x-1)/3) -- Same
and floor((c2.y-1)/3) = floor((c1.y-1)/3) -- square
and (c1.x <> c2.x or c1.y <> c2.y) -- Exclude self!
and c2.z like '%'||i||'%'
)
)
)
loop
update cells
set z = i
where x = rec.x
and y = rec.y;
-- Note how many cells processed on this pass
processed := processed+1;
end loop;
end loop;
-- Have we solved it yet?
select count(*) into cnt from cells where length(z) > 1 and rownum = 1;
exit when cnt = 0;
-- No. If we didn't achieve anything on this pass then give up
if processed = 0 then
raise it_failed;
end if;
end loop;
print;
dbms_output.put_line('SUCCESS!!!');
exception
when it_failed then
print;
dbms_output.put_line('FAILED!!!');
end;

procedure print
is
begin
for rec in (select * from cells order by x,y)
loop
dbms_output.put(rec.z||' ');
if rec.y = 9 then
dbms_output.put_line('');
end if;
end loop;
end;
end;
/

Now, how to use it. Let's take the following Su Doku puzzle as an example:
.6.1.4.5.
..83.56..
2.......1
8..4.7..6
..6...3..
7..9.1..4
5.......2
..72.69..
.4.5.8.7.


We can solve this as follows:
begin
cell_pkg.solve(' 6 1 4 5 83 56 2 18 4 7 6 6 3 7 9 1 45 2 72 69 4 5 8 7 ');
end;
/

(Enter all the grid values reading from left to right, top to bottom, including ALL spaces).
The output will look like this:
9 6 3 1 7 4 2 5 8
1 7 8 3 2 5 6 4 9
2 5 4 6 8 9 7 3 1
8 2 1 4 3 7 5 9 6
4 9 6 8 5 2 3 1 7
7 3 5 9 6 1 8 2 4
5 8 9 7 1 3 4 6 2
3 1 7 2 4 6 9 8 5
6 4 2 5 9 8 1 7 3
SUCCESS!!!


But perhaps you can do better? If you can write a better version, please let me know!

UPDATE 20 April 2006: Bill Magee has picked up the challenge and run with it, and his improved version is here.

Wednesday, August 24, 2005

What does GROUP BY CUBE do?

This is one of those "new" features that has actually been around for a long time, but I had never used it. Then recently I was asked to tune a long-running report, and it occured to me that it might just be what I needed.

The report consisted of 4 summary queries, each of which presented the same data but summarised by a different "dimension". Imagine it was based on the EMP table, then the report would show:

Employee Summary by Job
-----------------------

Job # Emps Total Sal
---------------------------------------- ---------- ----------
CLERK 4 4150
ANALYST 2 6000
MANAGER 3 8275
SALESMAN 4 5600
PRESIDENT 1 5000
---------- ----------
14 29025


Employee Summary by Manager
---------------------------

Mgr # Emps Total Sal
---------------------------------------- ---------- ----------
7566 2 6000
7698 5 6550
7782 1 1300
7788 1 1100
7839 3 8275
7902 1 800
---------- ----------
14 29025

Employee Summary by Department
------------------------------

Dept # Emps Total Sal
---------------------------------------- ---------- ----------
10 3 8750
20 5 10875
30 6 9400
---------- ----------
14 29025


That is only 3 queries, but you get the idea.

Each query took 10 minutes to run (reasonable given the complexity of the query and the amount of data), so the whole report took 40 minutes.

My goal was to reduce the time to more like 10 minutes - perhaps by pre-computing the results grouped by all 4 dimensions into a temporary table and then reporting on that. However, it occured to me that GROUP BY CUBE might be what I needed, so I tried it out. Here is a simple query:

SQL> select job, mgr, deptno, count(*) numemps, sum(sal) tots
2 from emp
3 group by cube(job, mgr, deptno);

JOB MGR DEPTNO NUMEMPS TOTSAL
--------- ---------- ---------- ---------- ----------
1 5000
14 29025
10 1 5000
10 3 8750
20 5 10875
30 6 9400
7566 2 6000
7566 20 2 6000
7698 5 6550
7698 30 5 6550
7782 1 1300
7782 10 1 1300
7788 1 1100
7788 20 1 1100
7839 3 8275
7839 10 1 2450
7839 20 1 2975
7839 30 1 2850
7902 1 800
7902 20 1 800
CLERK 4 4150
CLERK 10 1 1300
CLERK 20 2 1900
CLERK 30 1 950
CLERK 7698 1 950
CLERK 7698 30 1 950
CLERK 7782 1 1300
CLERK 7782 10 1 1300
CLERK 7788 1 1100
CLERK 7788 20 1 1100
CLERK 7902 1 800
CLERK 7902 20 1 800
ANALYST 2 6000
ANALYST 20 2 6000
ANALYST 7566 2 6000
ANALYST 7566 20 2 6000
MANAGER 3 8275
MANAGER 10 1 2450
MANAGER 20 1 2975
MANAGER 30 1 2850
MANAGER 7839 3 8275
MANAGER 7839 10 1 2450
MANAGER 7839 20 1 2975
MANAGER 7839 30 1 2850
SALESMAN 4 5600
SALESMAN 30 4 5600
SALESMAN 7698 4 5600
SALESMAN 7698 30 4 5600
PRESIDENT 1 5000
PRESIDENT 10 1 5000
PRESIDENT 10 1 5000

52 rows selected.


Buried in amongst all those rows is all the data I need. There is a GROUPING function that can be used to see which group a row belongs to:

SQL> select job, mgr, deptno, count(*) numemps, sum(sal) totsal
2 , grouping(job) gj, grouping(mgr) gm, grouping(deptno) gd
3 from emp
4 group by cube(job, mgr, deptno);

JOB MGR DEPTNO NUMEMPS TOTSAL GJ GM GD
--------- ---------- ---------- ---------- ---------- ---------- ---------- ----------
1 5000 1 0 1
14 29025 1 1 1
10 1 5000 1 0 0
10 3 8750 1 1 0
20 5 10875 1 1 0
30 6 9400 1 1 0
7566 2 6000 1 0 1
7566 20 2 6000 1 0 0
7698 5 6550 1 0 1
7698 30 5 6550 1 0 0
7782 1 1300 1 0 1
7782 10 1 1300 1 0 0
7788 1 1100 1 0 1
7788 20 1 1100 1 0 0
7839 3 8275 1 0 1
7839 10 1 2450 1 0 0
7839 20 1 2975 1 0 0
7839 30 1 2850 1 0 0
7902 1 800 1 0 1
7902 20 1 800 1 0 0
CLERK 4 4150 0 1 1
CLERK 10 1 1300 0 1 0
CLERK 20 2 1900 0 1 0
CLERK 30 1 950 0 1 0
CLERK 7698 1 950 0 0 1
CLERK 7698 30 1 950 0 0 0
CLERK 7782 1 1300 0 0 1
CLERK 7782 10 1 1300 0 0 0
CLERK 7788 1 1100 0 0 1
CLERK 7788 20 1 1100 0 0 0
CLERK 7902 1 800 0 0 1
CLERK 7902 20 1 800 0 0 0
ANALYST 2 6000 0 1 1
ANALYST 20 2 6000 0 1 0
ANALYST 7566 2 6000 0 0 1
ANALYST 7566 20 2 6000 0 0 0
MANAGER 3 8275 0 1 1
MANAGER 10 1 2450 0 1 0
MANAGER 20 1 2975 0 1 0
MANAGER 30 1 2850 0 1 0
MANAGER 7839 3 8275 0 0 1
MANAGER 7839 10 1 2450 0 0 0
MANAGER 7839 20 1 2975 0 0 0
MANAGER 7839 30 1 2850 0 0 0
SALESMAN 4 5600 0 1 1
SALESMAN 30 4 5600 0 1 0
SALESMAN 7698 4 5600 0 0 1
SALESMAN 7698 30 4 5600 0 0 0
PRESIDENT 1 5000 0 0 1
PRESIDENT 1 5000 0 1 1
PRESIDENT 10 1 5000 0 0 0
PRESIDENT 10 1 5000 0 1 0


Based on that, I can filter out the rows I'm not interested in and format the results like this:

SQL> select case when gj=0 and gm=1 and gd=1 then 'Job'
2 when gj=1 and gm=0 and gd=1 then 'Manager'
3 when gj=1 and gm=1 and gd=0 then 'Dept'
4 end as keytype
5 , case when gj=0 and gm=1 and gd=1 then job
6 when gj=1 and gm=0 and gd=1 then to_char(mgr)
7 when gj=1 and gm=1 and gd=0 then to_char(deptno)
8 end as keyval
9 , numemps, totsal
10 from
11 (
12 select job, mgr, deptno, count(*) numemps, sum(sal) totsal
13 , grouping(job) gj, grouping(mgr) gm, grouping(deptno) gd
14 from emp
15 group by cube(job, mgr, deptno)
16 )
17 where (gj=0 and gm=1 and gd=1)
18 or (gj=1 and gm=0 and gd=1)
19 or (gj=1 and gm=1 and gd=0);

KEYTYPE KEYVAL NUMEMPS TOTSAL
------- ---------------------------------------- ---------- ----------
Manager 7566 2 6000
7698 5 6550
7782 1 1300
7788 1 1100
7839 3 8275
7902 1 800
1 5000
******* ---------- ----------
sum 14 29025
Job ANALYST 2 6000
CLERK 4 4150
MANAGER 3 8275
PRESIDENT 1 5000
SALESMAN 4 5600
******* ---------- ----------
sum 14 29025
Dept 10 3 8750
20 5 10875
30 6 9400
******* ---------- ----------
sum 14 29025


... which gives me the breakdown I want in a single query, and provides the performance boost I was hoping for!

Tuesday, November 23, 2004

Vadim Tropashko on Complex Constraints

I am flattered to find that my first article here (Enforcing Complex Constraints in Oracle) is credited in the Acknowledgements of this interesting article by Vadim Tropashko on DBAZine:

http://www.dbazine.com/tropashko8.shtml

As well as approaching the subject from first principles, the article also suggest further applications of such constraints, e.g. "foreign keys" to more than one parent table, and enforcing uniqueness of column values across more than one table.

Friday, October 15, 2004

Parsing delimited fields in a character string

Often we have a need to parse a character string to get data from fields within it. Of course, SQL Loader handles this nicely, but sometimes we may be getting the data via a different route such as from a table or via UTL_FILE.

The following package facilitates this:

CREATE OR REPLACE PACKAGE parse AS
  /*
  || Package of utility procedures for parsing delimited or fixed position strings into tables
  || of individual values, and vice versa.
  */
  TYPE varchar2_table IS TABLE OF VARCHAR2(32767) INDEX BY BINARY_INTEGER;
  PROCEDURE delimstring_to_table
    ( p_delimstring IN VARCHAR2
    , p_table OUT varchar2_table
    , p_nfields OUT INTEGER
    , p_delim IN VARCHAR2 DEFAULT ','
    );
  PROCEDURE table_to_delimstring
    ( p_table IN varchar2_table
    , p_delimstring OUT VARCHAR2
    , p_delim IN VARCHAR2 DEFAULT ','
    );
END parse;
/
CREATE OR REPLACE PACKAGE BODY parse AS
  PROCEDURE delimstring_to_table
    ( p_delimstring IN VARCHAR2
    , p_table OUT varchar2_table
    , p_nfields OUT INTEGER
    , p_delim IN VARCHAR2 DEFAULT ','
    )
  IS
    v_string VARCHAR2(32767) := p_delimstring;
    v_nfields PLS_INTEGER := 1;
    v_table varchar2_table;
    v_delimpos PLS_INTEGER := INSTR(p_delimstring, p_delim);
    v_delimlen PLS_INTEGER := LENGTH(p_delim);
  BEGIN
    WHILE v_delimpos > 0
    LOOP
      v_table(v_nfields) := SUBSTR(v_string,1,v_delimpos-1);
      v_string := SUBSTR(v_string,v_delimpos+v_delimlen);
      v_nfields := v_nfields+1;
      v_delimpos := INSTR(v_string, p_delim);
    END LOOP;
    v_table(v_nfields) := v_string;
    p_table := v_table;
    p_nfields := v_nfields;
  END delimstring_to_table;
  PROCEDURE table_to_delimstring
    ( p_table IN varchar2_table
    , p_delimstring OUT VARCHAR2
    , p_delim IN VARCHAR2 DEFAULT ','
    )
  IS
    v_nfields PLS_INTEGER := p_table.COUNT;
    v_string VARCHAR2(32767);
  BEGIN
    FOR i IN 1..v_nfields
    LOOP
      v_string := v_string || p_table(i);
      IF i != v_nfields THEN
        v_string := v_string || p_delim;
      END IF;
    END LOOP;
    p_delimstring := v_string;
  END table_to_delimstring;
END parse;
/

This is how you might use it with a standard comma-delimited string:

SQL> declare
  2    v_tab parse.varchar2_table;
  3    v_nfields integer;
  4    v_string varchar2(1000) := '1000,Smith,John,13-May-1970';
  5  begin
  6    parse.delimstring_to_table (v_string, v_tab, v_nfields);
  7    for i in 1..v_nfields loop
  8      dbms_output.put_line('Field('||i||') = '||v_tab(i));
  9    end loop;
 10  end;
11 /
Field(1) = 1000
Field(2) = Smith
Field(3) = John
Field(4) = 13-May-1970

PL/SQL procedure successfully completed.

Enforcing complex constraints in Oracle

Oracle supports various kinds of declarative integrity constraints:
  • Primary Key: Uniquely identifies a row in the table
  • Unique: Other columns that must be unique
  • Foreign Key: Column value must match value in another table
  • Check: Simple single-table, single-row data rules.
Examples of possible check constraints are:
“start_date <= end_date”

“check_ind in (‘Y’,’N’)”
“amount between 0 and 99999.99”

However, many more complex business rules cannot be implemented via the above constraints. For example:

  • Employee Project Assignment start date and end date must fall between Project start date and end date
  • An employee may not have 2 Employee Project Assignments that overlap date-wise
  • A Project cannot have more than 10 Employees Assignments
  • An Employee cannot book time to a Task on a Project to which he is not assigned
  • The manager of a Department must belong to the Department.
These are usually enforced (if at all) procedurally, by one of the following methods:
  • Code in the application screens
  • Code in stored procedures (APIs) called from the application screens
  • Database triggers
These all have disadvantages compared to the declarative approach of constraints:
  • Code in application screens can be bypassed, and also the same rule must often be implemented in many places (e.g. in both the Project screen and the Assignment screen).
  • Code in stored procedures must also often be implemented in many places (e.g. in both the Project package and the Assignment package). Also, to prevent bypassing of the rules, all updates must be done via the package, which limits the functionality available to the user (cannot write ad hoc updates)
  • Code in triggers must also often be implemented in many places (e.g. in both the Project triggers and the Assignment triggers). Also, triggers can often become complex due to work-arounds to avoid mutating tables issues etc.
  • Any procedural solution must explicitly lock records to prevent corruption in a multi-user environment – e.g. if a user is amending an Employee Assignment, then no other user may be allowed to amend Employee Assignments for the same employee, or to amend the Project dates.
This paper shows how such complex constraints can be implemented declaratively, such that the rules are defined once and then applied to all updates from whatever source.

This began with an on-line discussion I had with Tom Kyte about enforcing complex constraints here: http://asktom.oracle.com/pls/ask/f?p=4950:8:::::F4950_P8_DISPLAYID:21389386132607

Objective

Ideally, given a complex rule such as one of the examples above, we would like to be able to create a complex check constraint (or “assertion” in ANSI SQL terms) such as:

CONSTRAINT c ON TABLE project p
CHECK (NOT EXISTS (SELECT null FROM emp_proj_assign ep
WHERE ep.projno = p.projno
AND (ep.start_date <> p.end_date)));

Such a constraint (if it could be implemented) would raise an error in any of the following situations:
  • User attempts to insert an emp_proj_assign record with dates outside the Project dates
  • User attempts to update an emp_proj_assign record with dates outside the Project dates
  • User attempts to update a Project record with dates that do not encompass all the associated emp_proj_assign dates
However, we can’t achieve that via constraints alone.

Solution: use Materialized Views

Materialized Views (a.k.a. Snapshots) are actually tables that are maintained by Oracle, whose contents correspond to the result of a query – i.e. like a view, but “materialized” because the result is actually stored in a table.

These provide the mechanism we need to implement the complex constraint as follows:
  • Create a materialized view to select data that violates the desired constraint (e.g. assignments where the dates are outside the associated project dates). The MV must be defined with REFRESH COMPLETE ON COMMIT so that it is updated before the end of the transaction.
  • Create a check constraint on the materialized view that always evaluates to FALSE – e.g. CHECK (1=0)
That’s it. Whenever the underlying tables are updated, the materialized view is refreshed. If the update violates the rule, then a row will be inserted into the materialized view; but the check constraint on the MV disallows any inserts into it, and so the transaction fails.

Issues
  • Oracle 8i cannot support REFRESH ON COMMIT on materialized views of the complexity required for some rules. 9i can handle some but not all (does not allow self-joins in MVs).. 10G can handle self-joins, but does not seem to allow subqueries. So this approach cannot be used for all rules.
  • Efficiency: needs to be benchmarked. Is a FAST refresh preferable to a COMPLETE refresh? With COMPLETE we have a query that looks at all rows of the MV’s base tables at the end of every transaction that affects those tables.
  • Cannot create a REFRESH ON COMMIT materialized view with a HAVING clause. For such cases (see example 3 below), the materialized view cannot include the constraint violation (e.g. HAVING COUNT(*) > 10), and so the check constraint must do it (e.g. CHECK (cnt <= 10)). Note that in this case the materialized view will consume space in the database.
Worked Examples

Based on the following tables:

create table project
( projno int primary key
, start_date date not null
, end_date date not null
);
create table emp_proj_assign
( empno int not null
, projno int not null
, start_date date not null
, end_date date not null
, primary key (empno, start_date)
);

1) Rule: An employee cannot have overlapping project assignments.

This is implemented as follows:

create materialized view emp_proj_mv1
refresh complete on commit as
select 1 dummy
from emp_proj_assign ep1, emp_proj_assign ep2
where ep1.empno = ep2.empno
and ep1.start_date <= ep2.end_date
and ep1.end_date >= ep2.start_date;


alter table emp_proj_mv1
add constraint emp_proj_mv1_chk
check (1=0) deferrable;

2) An employee's project assignment dates must fall between the project start
and end dates

create materialized view emp_proj_mv2
refresh complete on commit as
select 1 dummy
from emp_proj_assign ep, project p
where ep.projno = p.projno
and (ep.start_date <> p.end_date);
alter table emp_proj_mv2
add constraint emp_proj_mv2_chk
check (1=0) deferrable;

3) A project may not have more than 10 assignments (in total, ever!):

create materialized view emp_proj_mv3
build immediate
refresh complete on commit as
select projno, count(*) cnt
from emp_proj_assign
group by projno;
alter table emp_proj_mv3
add constraint emp_proj_mv3_chk
check (cnt <= 10)
deferrable;

4) A project cannot have more than 10 employees assigned at the same time.

(I have not yet worked this one out!)