Following a series of votes of no-confidence the Executive Mayor of Johannesburg was (again) removed from the post on Thursday January 26. On Friday a new Executive Mayor drawn from a tiny religious party (Al-Jamah) was appointed as the new mayor of the largest city in the country. Executive authority in the city now seems more fragile than ever before - the new mayor will be in office for only as long as rival coalitions are absent. Part deals, by-elections etc. may result in the new mayor being deposed at very short notice. As long as no political party has a clear majority in council the post of Executive Mayor will be insecure.
The post of Executive Mayor is extremely influential and changes in the mayorship results in the appointment of a new mayoral committee. More importantly the changes immediately impact on the objectives and priorities of the city. Consequently in all metros where coalitions are required Executive Mayors are insecure and service delivery is hostage to political interests.
The Executive Mayor system was adopted across virtually all the metropolitan municipalities where a single party has a clear majority. The system came to be entrenched in metros and smaller cities across the country. However as the influence of the ruling party waned giving rise to the need for coalitions the political instability that now typifies local government rose.
However at the heart of the current instability is not coalition politics itself. Bemoaning the need for coalitions is to condemn voters for reconsidering their political choices and to blame instability on their lack of political uniformity. Rather the problem is the attempt by metros to run coalitions under what is largely an authoritarian system that of the Executive Mayor.
Executive Mayors are authoritative figures which are enormously influential. Executive Mayors appoint members of the Mayoral Committee to assist them but are not held to account by that committee. Executive Mayors are meant to be held to account for how well the administration performs and they report back to the city council annually. For the most part they are afforded immense latitude in how they execute city affairs. This latitude and intermittent accountability makes the post of Executive Mayor pivotal to city business and makes it a post highly desirable to those seeking power and influence. This influence heightens the competition for the post - giving rise to the instability now evident.
Fortunately the Municipal Structures Act allows for metros to be run under one of two primary systems. The most prominent being that of Executive Mayor. The less familiar alternative is that of a "collective executive". Under this system the city is run by an executive committee. The executive committee assumes responsibility for managing city affairs and makes the important decisions - with the post of mayor being largely ceremonial. Council decisions are taken collectively. The legislation requires that the composition of the committee to reflect the composition of council (and it thus reflects residents voting patterns). This ensures that all the main political parties participate in executive authority (and prevents executive authority being vested in political parties without significant support).
Unlike under the Executive Mayor system the deliberations of the Collective Executives are open as is the way members vote. The Collective Executive thus promotes transparency and, consequently, we are able to hold representatives accountable. Moreover the collective system removes the incentive parties have to destabilise the administration. When you are party to decision making you have no incentive to propose a vote of no-confidence in the decision makers or replace the current executive.
In the past various Metros (Cape Town and eThekwini for example) have been run under Collective Executive systems. However this has always been when no one political party was dominant. When a party did become dominant it was expedient to revert to Executive mayorships.
Hopefully our elected representatives in Johannesburg (and Tshwane, Ekuruhleni, eThekwini, Nelson Mandela......) will soon choose to put aside their lust for power and start working as a collective.
Saturday, January 28, 2023
Political instability if S.A. metros easily addressed.
Wednesday, November 1, 2017
Database documentation in R
R database and process documentation
As someone who has to return to old R projects the management of 'legacy' databases is an on-going 'challenge'. Some analysis are frequently called on to replicated or modify previous work or to use old databases for new analysis. Knowing what is in a database, what changes have been made to it and how to repeat key processes is central to reusing those resources efficiently. Comprehensive documentation is obviously key to redeploying the databases efficiently but, candidly, documentation tends to be poor. Usually impatience coupled and the perception that analysis and procedures seemed perfectly clear, if not intuitive, at the time contributes to inadequate documentation. In practice returning to old databases usually results in considerable time being spent on deciphering .Rhistory scripts and divining what had been done previously. This invariably results in a few 'WTF' moments or total bewilderment when trying to reverse engineer what the earlier analyst did.
Fortunately a number of protocals and procedures can radically improve the maintenance of databases and scripts.
1. Keep databases (i.e. everything in the current R directory) small
By keeping databases small and focussed on key subjects it is easier to increase the level of documentation (see below) and eliminate the fluff that inevitably clutters working directories.
Be generous in setting the memory allocated to ".Rhistory". On linux systems this is set using a command in a system-wide or personal .Rprofile file. This file should contain a line like:
Sys.setenv(R_HISTSIZE='100000')
The size should be large enough to ensure all changes to that database are recorded. As .Rhistory is a record of all commands submitted keeping these files clean requires regular housekeeping. Ideally .Rhistory should have only the commands required to replicate the output ab initio. The .Rhistory file can easily be cluttered by repeated iterations, experimentation or the running of demo syntax. Users making minute adjustments on a ggplot, for example, can easily issue very similar ggplot( commands a few dozen times thereby cluttering the script file with fluff.
4. Use R through a script editor
To get better use from .Rhistory it is useful to use R through an editor like EMACS (vis ESS) or VI. These editors bring highlighting, colour coding, spell checking, formatting and other facilities to both .Rhistory and the current session. More importantly syntax editors bring search functionality into the active session of R and .Rhistory. The use of a script editor makes it easy to find every instance where a particular command or string was used or a particular variable referred to. Clearly this can be done by simply searching the .Rhistory but with slight modifications it is possible to rotate through every instance a particular command was used. The user can, for example, rotate through every ggplot(), lm() or table() command to find when it was last used and what the syntax was.
Under EMACS a slight modification to the .emacs file in the home directory is needed. These changes enable users to type the first few letters of a command and and then use the up and down arrows to rotate through useage. The following text must be place in a personal .emacs file:
---------------------------------------------------------------------
(eval-after-load
"comint"
'(progn
(setq comint-input-ring-size 20480)
(setq comint-scroll-to-bottom-on-output 'others) ; not current
;;=default: (setq comint-scroll-to-bottom-on-input nil)
(setq comint-scroll-show-maximum-output t) ;;; this is the key
(define-key comint-mode-map [up]
'comint-previous-matching-input-from-input)
(define-key comint-mode-map [down]
'comint-next-matching-input-from-input)
(Define-key comint-mode-map "\C-a" 'comint-bol)))
----------------------------------------------------------------
Now, by typing "ggplot(" at the prompt and pressing the up arrow EMACS will show, in order, every command starting with the string "ggplot(" showing when it was used. While this is useful to iterate through your history its utility is enhanced by adopting standardised practices that allow easy access to key functions.
Instead of using the standard format of LHS <- eg.="" i="" rhs="">comment(dfname) <- i="" nbsp="">'this dataframe was altered by adding column called "imp_val"' )
routinely reverse the direction of assignment to: "xx -> yyy". For example the above command becomes ...
comment(dfname) -> 'this dataframe was altered by adding column called "imp_val" '
Now if, in EMACS, the user types "comment(" and press the up arrow EMACS will rotate through every comment and (hopefully) find the relevant assignment. Analysts are more likely to be looking for statistical procedures or plots rather than comment assignment. If every import of csv data followed the syntax of "read.csv(" followed by the assignment " -> importedfilename" the user can easily rotate through all the csv imports. It now becomes easy to examine processes used in creating the current database. If the direction of assignment was not reversed and remains "LHS <- -="" i="" rhs=""> then pressing the up arrow will not find commands but assignment objects. For this to be of value the user will need to know the assigned name.
5. Create a sparse model of core commands
The practice of following the above syntax format also has the advantage of enabling the user to extract every assignment with ease. Typically analysts will experiment with a procedure or plot until the desired results are achieved. Once the user is happy with the syntax the result/output is assigned a name (by using the " -> newname" syntax).
If this syntax format is routinely used then every change in a value or assignment can be extracted from .Rhistory with ease. Under linux this key syntax can be parsed to a new file (called "mychanges.txt") using a command like:
less .Rhistory | grep -E ' -> |%>%' > mychanges.txt
If the user has been inserting #comments into the syntax file then use:
less .Rhistory | grep -E ' -> |%>%|#' > mychanges.txt
The file "mychanges.txt' is a record of every assignment made in that database using either "->" or tidyverse's "%>%". This new file is a complete record of every change made to that database and contains all the information needed to replicate all the work to date.
Commands are often refined incrementally until the desired outcome is reached and the extracted "mychanges.txt" file will contain every iteration made. In practice analyst usually only want the final iteration to be preserved (analysis presumably stops when the solution has been identified). It is a little extra housekeeping to remove all iterations but the last in "mychanges.txt". Doing this should result in a sparse and clean syntax file that records all changes to the database and no 'fluff'.
6. Routinely use a file to record notes
The key to adequate documentation is being reminded to make notes and being able to do so easily. Database documentation can be improved by adding a decriptive file to record intentions, procedures, progress or conclusions to every R directory (and thus to every R database). To be consistent keep one of these files in every R directory and use the same name eg. "desc.txt".
On linux systems the process is facilitated by creating an alias for EMACS (or VI) that equates the intial command "emacs" with "emacs desc.txt". Now when the user navigates to the relevant R database/directory typing "emacs"results in "desc.txt" being created (or re-opened) and opened in EMACS. The user can thus edit this file while running R in another buffer ("Ctrl-x 2" to open a new buffer in which to run R via "Alt-x R").
On Debian systems the alias is set in the home directory's .bashrc or .bash_aliases file using a string like:
alias emacs="emacs -g -1 desc.txt" (or put this into a script as shown below)
The "desc.txt" file is used to record notes that don't quite belong in .Rhistory. Once the R session is over exiting requires that EMACS be closed. Before the user exits EMACS they will be prompted to save changes to "desc.txt". This can be done after adding notes that will guide future users. As the notes made gradually accumulate a useful addition to .Rhistory is created.
7. Routinely add a timestamp to .Rhistory
Placing a timestamp in .Rhistory may help you to find previously used procedures or identify when modifications were made to a database. However placing a timestamp in .Rhistory is not straight-forward as this file is only updated when you exit R. Also .Rhistory records only the commands issued and not the output (the date etc.). One workaround is to create a script that appends a timestamp to .Rhistory immediately on starting R. I combine the insertion of the timestamp with the opening of "desc.txt" referred to above through a script.
To use the script navigate to the working directory and start "emacsR.sh". emacsR.sh is a text file which contains the following text
8. Refer to R objects in external products
The best record of how a database was used/applied is kept by writing reports in R Markdown, or knitr. This is because the code 'chunks' in the scripts refers to the R objects directly. Users, for example, can see that a particular table was used by crosstbulating variables x and y and what the syntax was used in creating a particular plot.
Unfortunately many clients insist on tables, graphics etc. being placed in a word processor document (eg. Word or LibreOffice) document. By doing this they can apply their own styles easily or not have to extend their IT skills. If you are required to do this rather than pasting plots and tables into their document try linking to the R object (OLE?) instead. This way you may be able to preserve references to both the name of the R object used and the database/directory it camefrom. These references help when trying to work out what the author (you?) did last time.
9. Full circle
The use of many small databases coupled to the creation of sparse histories and descriptive files may result in a plethora of files and greater confusion. Fortunately on linux systems it is easy enough to examine every .Rhistory (or 'desc.txt' file) for a particular search phrase. Once the directory has been identified the user has a reference to the relevant database and is better able to locate the key information on procedures or data.
An example of a process may run something like this:
A user needs to change a plot they provided in a report last year.
1. From item 8. above the report In LibreOffice/Word contains a table produced by R called, say, "fiscalcliff".
2. Search every R database on the system that produced an object called "fiscalcliff". This is done by typing, in linux,
find / -name .Rhistory -exec grep -H "fiscalcliff" {} \;
The '-H' command above outputs the filename of relevance. Now you should know which directory is most relevant.
3. Navigate to the database and open 'mychanges.txt' created from step 5 above (or just use .Rhistory). Search (based on step 4 above) from the end of the file backwards for 'fiscalcliff' ("Ctrl-r fiscalcliff" in EMACS) to show the last time that term was used.
4. The mychanges.txt file will contain all the commands needed to replicate everything to get to that point of output. Theoretically the user merely has to paste all the relevant text into R to get to that point again.
Caveats
There are procedures that may not be saved using the above text varbatim. Both base plots and ggplot, for example do not output graphics using the ' -> ' or ' <- assignment.="" base="" i="" like="" plots="" something="" use="">png('filename.png')
and ggplot uses ggsave('filename.png'). These commands can be outputted to the sparse history (mychanges.txt) file by widening the grep term in:
less .Rhistory | grep -E ' -> |%>%|#' mychanges.txt
to include the additional terms. For example ggplot users may want to use:
less .Rhistory | grep -E ' -> |%>%|#|ggsave' > mychanges.txt
As someone who has to return to old R projects the management of 'legacy' databases is an on-going 'challenge'. Some analysis are frequently called on to replicated or modify previous work or to use old databases for new analysis. Knowing what is in a database, what changes have been made to it and how to repeat key processes is central to reusing those resources efficiently. Comprehensive documentation is obviously key to redeploying the databases efficiently but, candidly, documentation tends to be poor. Usually impatience coupled and the perception that analysis and procedures seemed perfectly clear, if not intuitive, at the time contributes to inadequate documentation. In practice returning to old databases usually results in considerable time being spent on deciphering .Rhistory scripts and divining what had been done previously. This invariably results in a few 'WTF' moments or total bewilderment when trying to reverse engineer what the earlier analyst did.
Fortunately a number of protocals and procedures can radically improve the maintenance of databases and scripts.
1. Keep databases (i.e. everything in the current R directory) small
By keeping databases small and focussed on key subjects it is easier to increase the level of documentation (see below) and eliminate the fluff that inevitably clutters working directories.
Ideally issuing the ls() command should result in as sparse an output as possible. Using multiple small databases allows for information to be compartmentalised (and thus 'filed') it may results in confusion as to which database should be used in the first place. As far as possible start every project in its own directory.
2. Issue all commands from the CLI.
To record all changes and procedures in scripts only issue commands through the command line (LI), this way a record of changes and procedures are always at hand. If you exit R using q('yes') R records all command line instructions in the hidden ".Rhistory" file. Unless you make extensive use of drop-down menus and some spreadsheet-like facilities, this automatically creates a record of the commands used. By systematically recording commands database changes and procedures become replicable.
Many spreadsheet-type actions are not recorded and using commands like fix(dataframename) results in changes that are not documented. Accordingly these routines should be avoided and all changes are made by issuing a command on the CLI.
Be generous in setting the memory allocated to ".Rhistory". On linux systems this is set using a command in a system-wide or personal .Rprofile file. This file should contain a line like:
Sys.setenv(R_HISTSIZE='100000')
The size should be large enough to ensure all changes to that database are recorded. As .Rhistory is a record of all commands submitted keeping these files clean requires regular housekeeping. Ideally .Rhistory should have only the commands required to replicate the output ab initio. The .Rhistory file can easily be cluttered by repeated iterations, experimentation or the running of demo syntax. Users making minute adjustments on a ggplot, for example, can easily issue very similar ggplot( commands a few dozen times thereby cluttering the script file with fluff.
3. Use "comments" everywhere you can.
"Comments" enable users to keep track of observations they wish to have a record of. and using them often is key to keeping track of modifications. Comments can be pegged to R objects like dataframes and dataframe columns using syntax like:
'Dataframe altered on 3 May 2016 by adding the column "imp_val"' -> comment(dfname)
or, referencing a column in a dataframe,
'These are imputed values for the age variable assuming missing obs were C.A.R.' -> comment(dfname$imp_val)
To retrieve the comment assigned to a particular dataframe column use:
comment(dfname$imp_val)
The main problem with comments is that they are concealed and users have to tell R to show the comment associated with an object. In other words the user must know, or suspect, that a comment was assigned to the relevant object. If users do not routinely use comment assignment (and retrieval) the facility easily falls into disuse.
"Comments" enable users to keep track of observations they wish to have a record of. and using them often is key to keeping track of modifications. Comments can be pegged to R objects like dataframes and dataframe columns using syntax like:
'Dataframe altered on 3 May 2016 by adding the column "imp_val"' -> comment(dfname)
or, referencing a column in a dataframe,
'These are imputed values for the age variable assuming missing obs were C.A.R.' -> comment(dfname$imp_val)
To retrieve the comment assigned to a particular dataframe column use:
comment(dfname$imp_val)
The main problem with comments is that they are concealed and users have to tell R to show the comment associated with an object. In other words the user must know, or suspect, that a comment was assigned to the relevant object. If users do not routinely use comment assignment (and retrieval) the facility easily falls into disuse.
4. Use R through a script editor
To get better use from .Rhistory it is useful to use R through an editor like EMACS (vis ESS) or VI. These editors bring highlighting, colour coding, spell checking, formatting and other facilities to both .Rhistory and the current session. More importantly syntax editors bring search functionality into the active session of R and .Rhistory. The use of a script editor makes it easy to find every instance where a particular command or string was used or a particular variable referred to. Clearly this can be done by simply searching the .Rhistory but with slight modifications it is possible to rotate through every instance a particular command was used. The user can, for example, rotate through every ggplot(), lm() or table() command to find when it was last used and what the syntax was.
Under EMACS a slight modification to the .emacs file in the home directory is needed. These changes enable users to type the first few letters of a command and and then use the up and down arrows to rotate through useage. The following text must be place in a personal .emacs file:
---------------------------------------------------------------------
(eval-after-load
"comint"
'(progn
(setq comint-input-ring-size 20480)
(setq comint-scroll-to-bottom-on-output 'others) ; not current
;;=default: (setq comint-scroll-to-bottom-on-input nil)
(setq comint-scroll-show-maximum-output t) ;;; this is the key
(define-key comint-mode-map [up]
'comint-previous-matching-input-from-input)
(define-key comint-mode-map [down]
'comint-next-matching-input-from-input)
(Define-key comint-mode-map "\C-a" 'comint-bol)))
----------------------------------------------------------------
Now, by typing "ggplot(" at the prompt and pressing the up arrow EMACS will show, in order, every command starting with the string "ggplot(" showing when it was used. While this is useful to iterate through your history its utility is enhanced by adopting standardised practices that allow easy access to key functions.
Instead of using the standard format of LHS <- eg.="" i="" rhs="">comment(dfname) <- i="" nbsp="">'this dataframe was altered by adding column called "imp_val"' )
routinely reverse the direction of assignment to: "xx -> yyy". For example the above command becomes ...
comment(dfname) -> 'this dataframe was altered by adding column called "imp_val" '
Now if, in EMACS, the user types "comment(" and press the up arrow EMACS will rotate through every comment and (hopefully) find the relevant assignment. Analysts are more likely to be looking for statistical procedures or plots rather than comment assignment. If every import of csv data followed the syntax of "read.csv(" followed by the assignment " -> importedfilename" the user can easily rotate through all the csv imports. It now becomes easy to examine processes used in creating the current database. If the direction of assignment was not reversed and remains "LHS <- -="" i="" rhs=""> then pressing the up arrow will not find commands but assignment objects. For this to be of value the user will need to know the assigned name.
5. Create a sparse model of core commands
The practice of following the above syntax format also has the advantage of enabling the user to extract every assignment with ease. Typically analysts will experiment with a procedure or plot until the desired results are achieved. Once the user is happy with the syntax the result/output is assigned a name (by using the " -> newname" syntax).
If this syntax format is routinely used then every change in a value or assignment can be extracted from .Rhistory with ease. Under linux this key syntax can be parsed to a new file (called "mychanges.txt") using a command like:
less .Rhistory | grep -E ' -> |%>%' > mychanges.txt
If the user has been inserting #comments into the syntax file then use:
less .Rhistory | grep -E ' -> |%>%|#' > mychanges.txt
The file "mychanges.txt' is a record of every assignment made in that database using either "->" or tidyverse's "%>%". This new file is a complete record of every change made to that database and contains all the information needed to replicate all the work to date.
Commands are often refined incrementally until the desired outcome is reached and the extracted "mychanges.txt" file will contain every iteration made. In practice analyst usually only want the final iteration to be preserved (analysis presumably stops when the solution has been identified). It is a little extra housekeeping to remove all iterations but the last in "mychanges.txt". Doing this should result in a sparse and clean syntax file that records all changes to the database and no 'fluff'.
6. Routinely use a file to record notes
The key to adequate documentation is being reminded to make notes and being able to do so easily. Database documentation can be improved by adding a decriptive file to record intentions, procedures, progress or conclusions to every R directory (and thus to every R database). To be consistent keep one of these files in every R directory and use the same name eg. "desc.txt".
On linux systems the process is facilitated by creating an alias for EMACS (or VI) that equates the intial command "emacs" with "emacs desc.txt". Now when the user navigates to the relevant R database/directory typing "emacs"results in "desc.txt" being created (or re-opened) and opened in EMACS. The user can thus edit this file while running R in another buffer ("Ctrl-x 2" to open a new buffer in which to run R via "Alt-x R").
On Debian systems the alias is set in the home directory's .bashrc or .bash_aliases file using a string like:
alias emacs="emacs -g -1 desc.txt" (or put this into a script as shown below)
The "desc.txt" file is used to record notes that don't quite belong in .Rhistory. Once the R session is over exiting requires that EMACS be closed. Before the user exits EMACS they will be prompted to save changes to "desc.txt". This can be done after adding notes that will guide future users. As the notes made gradually accumulate a useful addition to .Rhistory is created.
7. Routinely add a timestamp to .Rhistory
Placing a timestamp in .Rhistory may help you to find previously used procedures or identify when modifications were made to a database. However placing a timestamp in .Rhistory is not straight-forward as this file is only updated when you exit R. Also .Rhistory records only the commands issued and not the output (the date etc.). One workaround is to create a script that appends a timestamp to .Rhistory immediately on starting R. I combine the insertion of the timestamp with the opening of "desc.txt" referred to above through a script.
To use the script navigate to the working directory and start "emacsR.sh". emacsR.sh is a text file which contains the following text
sed -i -e '$a\ ' .Rhistory
echo 'Session started.... ' $(date +%Y-%m-%d:%H:%M) >> .Rhistory
sed -i -e '$a\ ' .Rhistory
emacs -g -1 desc.txt
Line 2 inserts a timestamp at the end of the .Rhistory file in the working directory. Lines 1 and 3 pad that text with a blank line and the last line opens "desc.txt" with emacs. Remember to make emacsR.sh executable ("sudo chmod a+x emacsR.sh").
8. Refer to R objects in external products
The best record of how a database was used/applied is kept by writing reports in R Markdown, or knitr. This is because the code 'chunks' in the scripts refers to the R objects directly. Users, for example, can see that a particular table was used by crosstbulating variables x and y and what the syntax was used in creating a particular plot.
Unfortunately many clients insist on tables, graphics etc. being placed in a word processor document (eg. Word or LibreOffice) document. By doing this they can apply their own styles easily or not have to extend their IT skills. If you are required to do this rather than pasting plots and tables into their document try linking to the R object (OLE?) instead. This way you may be able to preserve references to both the name of the R object used and the database/directory it camefrom. These references help when trying to work out what the author (you?) did last time.
9. Full circle
The use of many small databases coupled to the creation of sparse histories and descriptive files may result in a plethora of files and greater confusion. Fortunately on linux systems it is easy enough to examine every .Rhistory (or 'desc.txt' file) for a particular search phrase. Once the directory has been identified the user has a reference to the relevant database and is better able to locate the key information on procedures or data.
An example of a process may run something like this:
A user needs to change a plot they provided in a report last year.
1. From item 8. above the report In LibreOffice/Word contains a table produced by R called, say, "fiscalcliff".
2. Search every R database on the system that produced an object called "fiscalcliff". This is done by typing, in linux,
find / -name .Rhistory -exec grep -H "fiscalcliff" {} \;
The '-H' command above outputs the filename of relevance. Now you should know which directory is most relevant.
3. Navigate to the database and open 'mychanges.txt' created from step 5 above (or just use .Rhistory). Search (based on step 4 above) from the end of the file backwards for 'fiscalcliff' ("Ctrl-r fiscalcliff" in EMACS) to show the last time that term was used.
4. The mychanges.txt file will contain all the commands needed to replicate everything to get to that point of output. Theoretically the user merely has to paste all the relevant text into R to get to that point again.
Caveats
There are procedures that may not be saved using the above text varbatim. Both base plots and ggplot, for example do not output graphics using the ' -> ' or ' <- assignment.="" base="" i="" like="" plots="" something="" use="">png('filename.png')
and ggplot uses ggsave('filename.png'). These commands can be outputted to the sparse history (mychanges.txt) file by widening the grep term in:
less .Rhistory | grep -E ' -> |%>%|#' mychanges.txt
to include the additional terms. For example ggplot users may want to use:
less .Rhistory | grep -E ' -> |%>%|#|ggsave' > mychanges.txt
Monday, February 15, 2016
Projections for 2016 local government elections
Before
the promises are made, campaigning starts and voter–behaviour
surveys begin the only way to predict elections outcomes hold is to
examine the past behaviour of voters. The projection of past patterns gives insight
into what South Africans can expect when the local government
election takes place in the third quarter of 2016. While projections offers an indication as what to expect from the election it should be
borne in mind that the objective of election campaigns is largely to
alter the expected outcome.
A
clear signal of voter intentions for 2016 was already given by the
2014 National/ Provincial election. By extrapolating the changes in
voting patterns between the last local government election (2011) and
2014 we can predict that some changes are in the offing for 2016. In
particular the ANC can expect declines of about four percent support
in Gauteng, the North West province and the Eastern Cape. These
declines will be partly offset by a similar percentage gain in ANC
vote share in kwaZulu-Natal. The largest movements in votes will be
evident in the cities.
In
general the ANC can expect to receive a significantly reduced
proportion of the votes cast in metropolitan areas and several other
cities. As far back as the 2014 National/ Provincial elections the
ANC received less than half the proportional representation (PR)
votes cast in Nelson Mandela Bay. As that was not a municipal
election it did not make any difference to the composition of the
metropolitan council then. The changes in support profile are likely
to impact on the composition of the municipal council now.
The
forecasts indicate that the ANC, while retaining its minority vote
share in Cape Town, will lose the absolute majority it currently
holds in:
- Nelson Mandela Bay,
- Tshwane and
- Johannesburg.
In
these three metropoles the ANC will continue to be the single largest
political party but will need the support of opposition councillors
to set budgets, pass motions, and enact by-laws. After the election
the ANC will hold an absolute majority in only four of the eight
metropoles.
The
ANC is also likely to lose its absolute majority in other
municipalities which are highly urbanised. These include:
- Randfontein,
- Kouga,
- Thabazimbi and
- Rustenburg.
In
Rustenburg the ANC majority, which stood at over three-quarters of
votes cast before 2011, has since been whittled away by the
opposition – in particular by the EFF.
However
the election also portends important gains for the ANC particularly
in kwaZulu-Natal. Overall the ANC is set to gain absolute majorities
in another 28 municipalities. In almost all of the 28 municipalities
the ANC is currently the largest political party, albeit one with
less than 50% of popular support. After 2016 election the ANC may
well no longer need the support of the opposition to pass budgets and
by-laws. While most of the 28 municipalities are in kwaZulu-Natal a
handful are in the Western and Northern Cape.
In
six of the 28 municipalities the ANC stands to win the municipality
from another political party. These include winning (from the IFP):
- Nkandla and
- Mthonjeni.
Further
south the ANC seems to win several municipalities currently dominated
by the DA. These are:
- Bitou,
- Laingsburg,
- Hessequa and
- Witzenberg.
Exactly
how the changing vote patterns impact on the composition of elected
councils depends on both how the PR votes are apportioned between
wards and the second leg of the election (the election of ward
councillors directly). It seems that while a marginal change in the
political alignment of larger cities is indicated these changes may
have a deep impact on the quality of their governance. Until then
each of the political parties will strive to ensure that the outcomes
are better than what is expected now.
Detailed results are available at: Projections of ANC support in 2016 municipal elections.
Saturday, April 26, 2014
The real inflation rate
A perennial point of contention among South Africans is the disjuncture between the rate of inflation stated by officials and economists and their lived experience. Individuals increasingly suggest that the "real" rate of inflation is substantially higher than what, for example, StatsSA tells us it is. To support their claims individuals are usually able to cite innumerable examples as to why their experiences debunk the official CPI (currently at about 6%).
Economists, in retort, point that the CPI and other estimates of inflation are scientifically based and drawn from rigorously collected data and known consumption patterns A great deal of effort is invested by authorities in collecting prices for a predefined basket of goods that mirrors what is known about consumers consumption patterns and purchasing habits. The inflation rate is derived from objective empirical studies that cannot easily be discounted. Pundits then invariably proceed to point out that mismatches are possible as the consumption patterns are unique to individuals while the CPI measures the aggregate impact.
They further point out that critics of the official rate cite price increases that are most striking while they remain silent on those products whose prices are rising slowly or even falling. Cellphone charges and the internet bandwith costs have, for example, plummeted just as demand for these utilities become increasingly important.
However these responses indicate lack of insight - the (supposedly subjective) experience of the critics are not necessarily incompatible with the (objective) official statistics.
On many items consumers are confronted with price increases that significantly exceed the inflation rate. In recent months these items have come to include food and fuel. In addition administered prices (i.e. prices that are set by state institutions who do not face competition for their services) have also risen far faster than the CPI. Included among the latter are municipal rates, the price of electricity and road tolls. One feature common to these products is that there are few or no substitutes and consumers can do relatively little to circumvent them. To ensure there is food on the table, they are able to get to work and their municipal rates are paid individuals have to bear with these increases and cut costs elsewhere. The only place where costs can be cut are on discretionary items like IT and cellphone costs. It is among the latter category that price increases have been most modest.
In other words consumers are unable to take full advantage of stagnant or declining prices as their income is absorbed by the rapidly increasing cost of essentials and administered prices. In the process their cost profile rises and their consumption patterns alter. The basket of goods that they consume changes with prices with an ever increasing proportion of the basket being made up of high inflation essentials.
In the interim the economists are deriving the CPI on a stagnant basket of goods - not one where the 20% increase in the cost of food and petrol and electricity is offset by commensurate reduction in consumption elsewhere. The composition of the basket of goods used in deriving the CPI is only reconsidered ever ten years or so. In reality households have to adjust their consumption patterns every time there are sustained increases in prices and not every ten years or so.
So, dear consumer, you may not be losing your grasp on reality. Your personal rate of inflation may be well into the double digits and it is cold comfort that the price of items you can no longer afford are rising less rapidly. On the other hand I suggest that the official statistics will have ever less relevance to individual consumers.
Economists, in retort, point that the CPI and other estimates of inflation are scientifically based and drawn from rigorously collected data and known consumption patterns A great deal of effort is invested by authorities in collecting prices for a predefined basket of goods that mirrors what is known about consumers consumption patterns and purchasing habits. The inflation rate is derived from objective empirical studies that cannot easily be discounted. Pundits then invariably proceed to point out that mismatches are possible as the consumption patterns are unique to individuals while the CPI measures the aggregate impact.
They further point out that critics of the official rate cite price increases that are most striking while they remain silent on those products whose prices are rising slowly or even falling. Cellphone charges and the internet bandwith costs have, for example, plummeted just as demand for these utilities become increasingly important.
However these responses indicate lack of insight - the (supposedly subjective) experience of the critics are not necessarily incompatible with the (objective) official statistics.
On many items consumers are confronted with price increases that significantly exceed the inflation rate. In recent months these items have come to include food and fuel. In addition administered prices (i.e. prices that are set by state institutions who do not face competition for their services) have also risen far faster than the CPI. Included among the latter are municipal rates, the price of electricity and road tolls. One feature common to these products is that there are few or no substitutes and consumers can do relatively little to circumvent them. To ensure there is food on the table, they are able to get to work and their municipal rates are paid individuals have to bear with these increases and cut costs elsewhere. The only place where costs can be cut are on discretionary items like IT and cellphone costs. It is among the latter category that price increases have been most modest.
In other words consumers are unable to take full advantage of stagnant or declining prices as their income is absorbed by the rapidly increasing cost of essentials and administered prices. In the process their cost profile rises and their consumption patterns alter. The basket of goods that they consume changes with prices with an ever increasing proportion of the basket being made up of high inflation essentials.
In the interim the economists are deriving the CPI on a stagnant basket of goods - not one where the 20% increase in the cost of food and petrol and electricity is offset by commensurate reduction in consumption elsewhere. The composition of the basket of goods used in deriving the CPI is only reconsidered ever ten years or so. In reality households have to adjust their consumption patterns every time there are sustained increases in prices and not every ten years or so.
So, dear consumer, you may not be losing your grasp on reality. Your personal rate of inflation may be well into the double digits and it is cold comfort that the price of items you can no longer afford are rising less rapidly. On the other hand I suggest that the official statistics will have ever less relevance to individual consumers.
Friday, December 20, 2013
Quality of governance and voting
AG reports and governance
Each year every state
department is measured in terms of the extent they have adhered to
the laws and regulations laid down with respect to accounting
procedures, financial practice, asset control, awarding of contracts
and so on. These assessments by the Auditor General (A) speak to the
extent to which prescribed minimum standards of governance are met as
they present a concise assessment of, inter alia, the levels
of wastage and corruption in state departments. As they deliver a
seemingly ``objective'' opinion of government performance and are an
invaluable measurement of the potential effectiveness of state
agencies. The AGs reports on municipal government are of particular
interest as they indicate why service levels targets may not have
been met, budgets went unspent or how much money was misappropriated
or wasted.
Municipalities obtain
"clean" audits by being able to demonstrate that they have
systematically adhered to the regulations and legislation that govern
their functioning. Being able to demonstrate this depends on several
factors including the competence of accounting officers, the adequacy
of IT and HR systems and presence of effective disincentives for
financial abuse. Although service delivery depends heavily of local
government performance it is at this tier of government that these
factors tend to be least adequate. The MLGI Audit Barometer, for
example, shows that 60 percent of local municipalities are
substantially in breach of the prescribed regulations and
legislation. Moreover, the more optimistic projections indicate that
at the current rate of improvement it will be more than a decade
before all local municipalities receive "unqualified"
audits i.e. audits where substantial deviations from required
practice cannot be identified.
Poor audit outcomes are
often explained in terms of the lack of suitably qualified staff. The
appointment of staff with the required technical competence and the
ability to ensure conformance with prescribed policies is essential
to achieving a positive audit. It would seem, prima facie, that large
towns and cities would be best positioned to attract qualified staff.
While this is manifest by most metropolitan municipalities achieving
good audit outcomes the view is undermined by the fact that, of all
the provinces, the largely rural KwaZulu-Natal local municipalities
achieves the best audit outcomes. The MLGI Audit Barometer shows that
83 percent of local municipalities in that province routinely obtain
unqualified audits. KwaZulu-Natal is closely followed by the Western
Cape where 80 percent of municipalities get clean audits. By contrast
in highly urbanised Gauteng 40% of the local and metropolitan
municipalities do not routinely get unqualified audits. Similarly
many remote rural rural municipalities like uMzimkhulu and Impendle
obtain excellent audit outcomes. Obviously the inability to attract
suitable personnel is not a key determinant of audit outcomes and
reasons for the massive variations in audit outcome thus have to be
sought elsewhere.
Voting patterns
In democratic electoral
systems determining who fills key administrative positions depends on
how much sway political parties hold in the constituency.
Consequently elections play a pivotal role in determining who fills
key position and, in turn, the quality of service delivery and the
diligence with which the rules and regulations are adhered to. In the
last local government election in 2011 the ruling party gained
controlled of over 85 percent of the 234 local municipalities. Given
the predominance of the ANC and the massive variation in the local
government audit outcomes no obvious trend between voting patterns
and audit outcomes is apparent.
An analysis of voting
returns and audit outcomes for 2011/12 reveals that municipalities
led by opposition parties (the DA, IFP, NFP etc.) received notably
better audit outcomes than ANC led municipalities. On the face of
municipalities in the opposition-controlled Western Cape are more
likely to receive a positive audit than municipalities in other
provinces. However, as indicated above, the best audit outcomes are
in kwaZulu-Natal where 80 percent of the local municipalities are
controlled by the ruling party. In that province there is little
reason why IFP or NFP dominated municipalities should have better
audit outcomes than ANC led municipalities. None of these
municipalities boast the qualities that would predispose them to
positive audit outcomes any more than ANC controlled municipalities
in kwaZulu-Natal.
While opposition led
municipalities do obtain better audit outcomes this does not seem to
be the result of an opposition party being elected. Good audit
outcomes are apparent in as many municipalities led by the ruling
party. What a correlation between audit outcomes and voting returns
does show is that any municipality dominated by a single political
party predisposes it to poor audit outcomes. Conversely, not being
dominated by a single political party systematically improves audit
outcomes. The graphic below illustrates this. It contrasts the
distribution of votes obtained by the wining party in
- municipalities that received an unqualified audit with
- those that received a qualified (or worse) audit.
It is clear that in
those municipalities with qualified audits the winning party
typically received about 75% of the total vote. In the vast majority
of "qualified" audit municipalities the winning party
received at least two-thirds of all votes cast in the 2011 local
government elections.
By contrast the winning
party in municipalities who got unqualified audits routinely received
only about 55% percent of the votes cast. The area where coalitions
are required as no one party gained an absolute majority are shown by
the lightly shaded area. This shows that only a small percentage of
municipalities run by coalitions receive qualified audits. It is
seemingly strong political dominance by a single party at the polls
that predispose the municipality to poor audit outcomes.
The general trend is:
- If a single party wins 90 percent of the vote that municipality has virtually no chance of getting an unqualified audit.
- If two-thirds of the population votes for one party then the municipality has less than a one-in-three chance of getting an unqualified audit.
This later situation
approximates the current national situation where the ANC has 66% of
the vote share and 30% of municipalities get unqualified audits. If
the winning party in a municipality was to get a simple majority (50%
of the vote) then the municipality has twice as much chance of
getting an unqualified audit.
The kwaZulu-Natal
experience in particular shows that audit quality is less influenced
by which political party is elected and more by how much competition
that political party faces. Moreover this trend bears out
irrespective of which political party is dominant. This
interpretation indicates that if the ruling party maintained control
over the overwhelming majority of municipalities (say the current
85%) audit quality would be improved if its share of votes declined
in those municipalities. The same goes for the opposition parties
that win municipalities (but no opposition party has a share of votes
that remotely approximates the ANC national share of votes).
This relationship
indicates that the voters have much to do with audit outcomes and
thus directly influence governance quality. However the result has
less to do with changing which political party runs the municipality
and more to do with ensuring that there is a healthy counter to
political dominance. Having a strong opposition ensures greater
scrutiny of activities and policies, improved transparency and,
ultimately, better quality of governance. Voters can thus improve
audit outcomes (and governance quality) even by voting for parties
that have no chance of winning the election.
There has been a
tendency for disaffected voters in South Africa to abstain from
voting. However this results only is a race to the bottom as party
performance in elections then depends on who loses the least number
of supporters. This analysis shows that voters can improve governance
quality by transferring their vote to another party - even one
destined to lose. The one key question that has yet to answered is
whether or not voters do in fact reward performance at the polls.
Wednesday, December 5, 2012
Zuma and the rise of the right.
At the last general election the ANC, recently placed under the stewardship of Jacob Zuma, was returned to power with the mandate of another 2 million voters. A
t the time pundits attributed the increased support to trade union and the ANC Youth League support for Zuma. However since that election there has been a funda
mental realignment within the ANC which should become readily apparent in Mangaung next month. Those factions attributed with giving the ANC its increased mandate are now among those most opposed to Zuma being elected for a second term. By contrast those supporting Zuma are a strange mixture of "radicals" and conservatives including MK veterans, religious leaders, the SACP and traditional leaders.
The realignment does not represent Zumas betrayal of his backers. Rather it reflects the recognition that the additional votes came primarily from "tribal" areas. At last count approximately one-quarter of South Africans of voting age resided in what are now referred to as "traditional" areas. This makes this constituency one of the most influential in the country and in the ANC.
The features that unite Zuma supporters are becoming ever more apparent. Not only are these groups socially conservative, they tend to be highly patriarchal and, at heart, anti-democratic. To gain the support of these groups candidates have to focus on beguiling their leaders rather than the rank and file. Zuma has to appeal not to the soldiers, congregants and subjects but to an elite of generals, bishops and kings. Obviously this calls for a fundamentally different approach to the populism of the Youth League we grew accustomed to.
Zuma's pursuit of support from traditional leaders is shown by the ANC's dogged pursuit of "traditional courts". For some time the Department of Justice has been attempting to pass legislation which grants tribal chiefs the right to try criminal and civil cases and impose sanctions as they see appropriate. The procedures and sanctions adopted in these courts are to accord with traditional law and cultural practices.
The Traditional Courts Bill has stalled on repeated rejection by the affected communities. While the media has tended to emphasise the criticism voiced by women, opposition to the bill is widespread. In the public hearings on the bill men have tended to dominate proceedings both numerically and in vociferousness. They dismiss the bill with as much vigour as that attributed to women. Despite the opposition (which extends into cabinet) the Zuma administration continues to promote the legislation.
Despite the opposition and its obvious unconstitutionality the bill being pushed through because of the advantages it offers traditional leaders. In return traditional leaders can deliver the support of their subjects - at both Mangaung and at the ballot box.
The ANC will probably deny that the phalanx of legislation focussing on traditional areas entrenches partriarchal and anti-egalitarian systems. Admittedly there are now more women on Traditional Councils. Legislation requires that 30 percent of representatives are women. This is all well and good except for the fact that there is no popular support for Traditional Councils and these Councils have no power to speak of. Not only do these elections not meet the criteria for being free-and-fair turnout in Traditional Council elections typically run at less than ten percent of the eligible population. The reason for the low turnout lies in the negligible influence elected representatives have in the Councils. Most people on the council are not elected but are appointed by the tribal leaders. Moreover, just as traditional courts are not compelled to observe the precepts of natural justice, the Traditional Councils are not required to adhere to the basic principles of democratic practice. Chiefs are not bound by the majority vote of the Councils. Given this it is surprising that so many people bother to vote in Traditional Council elections at all.
Constitutional democrats may well have felt relief when the populist Youth Leagues fell from Zuma's grace. Current tension between the ANC and unions may even comfort those who fear the influence of the left within the ANC. However the alignment of the ANC leader with anti-democratic factions will prove to be far more worrisome. When he resebtly addressed the National House of Traditional Leaders Zuma encouraged them to reject "white mans justice". His justification rests on the view that "white mans justice" turns on cold facts rather than the "warm bodies" prioritised by indigenous justice systems. Zuma echoed criticisms of the judicial system as retributive, overly codified and "un-african". However the constitution now forms the bedrock of that "white mans justice". This perspective coupled with the bulldozing of the Traditional Courts and entrenching the undemocractic Traditional Councils on the most vulnerable communities in the country suggest that Zuma may be similarly predisposed to dismissing the constitution. He seems ready to court traditional authorities in the name of "Africanness" regardless of how it undermines democracy or how devisive it is within the ANC.
t the time pundits attributed the increased support to trade union and the ANC Youth League support for Zuma. However since that election there has been a funda
mental realignment within the ANC which should become readily apparent in Mangaung next month. Those factions attributed with giving the ANC its increased mandate are now among those most opposed to Zuma being elected for a second term. By contrast those supporting Zuma are a strange mixture of "radicals" and conservatives including MK veterans, religious leaders, the SACP and traditional leaders.
The realignment does not represent Zumas betrayal of his backers. Rather it reflects the recognition that the additional votes came primarily from "tribal" areas. At last count approximately one-quarter of South Africans of voting age resided in what are now referred to as "traditional" areas. This makes this constituency one of the most influential in the country and in the ANC.
The features that unite Zuma supporters are becoming ever more apparent. Not only are these groups socially conservative, they tend to be highly patriarchal and, at heart, anti-democratic. To gain the support of these groups candidates have to focus on beguiling their leaders rather than the rank and file. Zuma has to appeal not to the soldiers, congregants and subjects but to an elite of generals, bishops and kings. Obviously this calls for a fundamentally different approach to the populism of the Youth League we grew accustomed to.
Zuma's pursuit of support from traditional leaders is shown by the ANC's dogged pursuit of "traditional courts". For some time the Department of Justice has been attempting to pass legislation which grants tribal chiefs the right to try criminal and civil cases and impose sanctions as they see appropriate. The procedures and sanctions adopted in these courts are to accord with traditional law and cultural practices.
The Traditional Courts Bill has stalled on repeated rejection by the affected communities. While the media has tended to emphasise the criticism voiced by women, opposition to the bill is widespread. In the public hearings on the bill men have tended to dominate proceedings both numerically and in vociferousness. They dismiss the bill with as much vigour as that attributed to women. Despite the opposition (which extends into cabinet) the Zuma administration continues to promote the legislation.
Despite the opposition and its obvious unconstitutionality the bill being pushed through because of the advantages it offers traditional leaders. In return traditional leaders can deliver the support of their subjects - at both Mangaung and at the ballot box.
The ANC will probably deny that the phalanx of legislation focussing on traditional areas entrenches partriarchal and anti-egalitarian systems. Admittedly there are now more women on Traditional Councils. Legislation requires that 30 percent of representatives are women. This is all well and good except for the fact that there is no popular support for Traditional Councils and these Councils have no power to speak of. Not only do these elections not meet the criteria for being free-and-fair turnout in Traditional Council elections typically run at less than ten percent of the eligible population. The reason for the low turnout lies in the negligible influence elected representatives have in the Councils. Most people on the council are not elected but are appointed by the tribal leaders. Moreover, just as traditional courts are not compelled to observe the precepts of natural justice, the Traditional Councils are not required to adhere to the basic principles of democratic practice. Chiefs are not bound by the majority vote of the Councils. Given this it is surprising that so many people bother to vote in Traditional Council elections at all.
Constitutional democrats may well have felt relief when the populist Youth Leagues fell from Zuma's grace. Current tension between the ANC and unions may even comfort those who fear the influence of the left within the ANC. However the alignment of the ANC leader with anti-democratic factions will prove to be far more worrisome. When he resebtly addressed the National House of Traditional Leaders Zuma encouraged them to reject "white mans justice". His justification rests on the view that "white mans justice" turns on cold facts rather than the "warm bodies" prioritised by indigenous justice systems. Zuma echoed criticisms of the judicial system as retributive, overly codified and "un-african". However the constitution now forms the bedrock of that "white mans justice". This perspective coupled with the bulldozing of the Traditional Courts and entrenching the undemocractic Traditional Councils on the most vulnerable communities in the country suggest that Zuma may be similarly predisposed to dismissing the constitution. He seems ready to court traditional authorities in the name of "Africanness" regardless of how it undermines democracy or how devisive it is within the ANC.
Tuesday, November 6, 2012
========================================================
The results of the 2011 South African census were release in November 2011. The top-line results revealed a few surprises including a higher than expected population growth rate. The population total of 51.7 million was 2.5 percent greater than StatsSA's mid-year population estimates for that year. This indicates a population 1.2 million more than anticipated.
Apparently the census also indicates an increase in the Total Fertility Rate (TFR) which rose to 2.7 from the 2.5 of a decade earlier. Apparently an increase in TFR typified all the major population groups classified in the census. While an eight percent increase in TFR may not appear unduly worrisome it actually has has profound implications for our understanding of development. In terms of Demographic Transition Theory an increase in TFR (and by implication in the IMR) indicates a society under great stress. Recent examples of such increase include war-time Afghanistan and Zambia under the IMF structural adjustment programme. The IMF's structural adjustment programmes greatly restricted access to health services in Zambia allowing for massive increased in mortality and fertility rates.
According to StatsSA the past ten years in South Africa has been marked by just such trauma. Part of the trauma is related to the impact of HIV/AIDS on the dominant population group. HIV/AIDS may have resulted in the observed increase in mortality rates. In response families were prompted to having a greater number of children giving rise to higher TFR.
However the data revealed by StatsSA shows that the increase in family size has primarily taken place in the past past five years. The graphic below shows the marked increase in the youngest five year age cohort.
The last five years has coincided with the period in which the impact of HIV/AIDS was greatly reduced by widespread uptake of AVRs. HIV/Aids thus offers an unconvincing argument for the increase in TFR.
The results of the census thus either confirm that a) South African society has been under such profound stress that the Demographic Transition has been reversed or b) a more mundane answer lies in the quality of StatsSA's tallying.
Subscribe to:
Posts (Atom)
