Following a series of votes of no-confidence the Executive Mayor of Johannesburg was (again) removed from the post on Thursday January 26. On Friday a new Executive Mayor drawn from a tiny religious party (Al-Jamah) was appointed as the new mayor of the largest city in the country. Executive authority in the city now seems more fragile than ever before - the new mayor will be in office for only as long as rival coalitions are absent. Part deals, by-elections etc. may result in the new mayor being deposed at very short notice. As long as no political party has a clear majority in council the post of Executive Mayor will be insecure.
The post of Executive Mayor is extremely influential and changes in the mayorship results in the appointment of a new mayoral committee. More importantly the changes immediately impact on the objectives and priorities of the city. Consequently in all metros where coalitions are required Executive Mayors are insecure and service delivery is hostage to political interests.
The Executive Mayor system was adopted across virtually all the metropolitan municipalities where a single party has a clear majority. The system came to be entrenched in metros and smaller cities across the country. However as the influence of the ruling party waned giving rise to the need for coalitions the political instability that now typifies local government rose.
However at the heart of the current instability is not coalition politics itself. Bemoaning the need for coalitions is to condemn voters for reconsidering their political choices and to blame instability on their lack of political uniformity. Rather the problem is the attempt by metros to run coalitions under what is largely an authoritarian system that of the Executive Mayor.
Executive Mayors are authoritative figures which are enormously influential. Executive Mayors appoint members of the Mayoral Committee to assist them but are not held to account by that committee. Executive Mayors are meant to be held to account for how well the administration performs and they report back to the city council annually. For the most part they are afforded immense latitude in how they execute city affairs. This latitude and intermittent accountability makes the post of Executive Mayor pivotal to city business and makes it a post highly desirable to those seeking power and influence. This influence heightens the competition for the post - giving rise to the instability now evident.
Fortunately the Municipal Structures Act allows for metros to be run under one of two primary systems. The most prominent being that of Executive Mayor. The less familiar alternative is that of a "collective executive". Under this system the city is run by an executive committee. The executive committee assumes responsibility for managing city affairs and makes the important decisions - with the post of mayor being largely ceremonial. Council decisions are taken collectively. The legislation requires that the composition of the committee to reflect the composition of council (and it thus reflects residents voting patterns). This ensures that all the main political parties participate in executive authority (and prevents executive authority being vested in political parties without significant support).
Unlike under the Executive Mayor system the deliberations of the Collective Executives are open as is the way members vote. The Collective Executive thus promotes transparency and, consequently, we are able to hold representatives accountable. Moreover the collective system removes the incentive parties have to destabilise the administration. When you are party to decision making you have no incentive to propose a vote of no-confidence in the decision makers or replace the current executive.
In the past various Metros (Cape Town and eThekwini for example) have been run under Collective Executive systems. However this has always been when no one political party was dominant. When a party did become dominant it was expedient to revert to Executive mayorships.
Hopefully our elected representatives in Johannesburg (and Tshwane, Ekuruhleni, eThekwini, Nelson Mandela......) will soon choose to put aside their lust for power and start working as a collective.
Saturday, January 28, 2023
Political instability if S.A. metros easily addressed.
Wednesday, November 1, 2017
Database documentation in R
R database and process documentation
As someone who has to return to old R projects the management of 'legacy' databases is an on-going 'challenge'. Some analysis are frequently called on to replicated or modify previous work or to use old databases for new analysis. Knowing what is in a database, what changes have been made to it and how to repeat key processes is central to reusing those resources efficiently. Comprehensive documentation is obviously key to redeploying the databases efficiently but, candidly, documentation tends to be poor. Usually impatience coupled and the perception that analysis and procedures seemed perfectly clear, if not intuitive, at the time contributes to inadequate documentation. In practice returning to old databases usually results in considerable time being spent on deciphering .Rhistory scripts and divining what had been done previously. This invariably results in a few 'WTF' moments or total bewilderment when trying to reverse engineer what the earlier analyst did.
Fortunately a number of protocals and procedures can radically improve the maintenance of databases and scripts.
1. Keep databases (i.e. everything in the current R directory) small
By keeping databases small and focussed on key subjects it is easier to increase the level of documentation (see below) and eliminate the fluff that inevitably clutters working directories.
Be generous in setting the memory allocated to ".Rhistory". On linux systems this is set using a command in a system-wide or personal .Rprofile file. This file should contain a line like:
Sys.setenv(R_HISTSIZE='100000')
The size should be large enough to ensure all changes to that database are recorded. As .Rhistory is a record of all commands submitted keeping these files clean requires regular housekeeping. Ideally .Rhistory should have only the commands required to replicate the output ab initio. The .Rhistory file can easily be cluttered by repeated iterations, experimentation or the running of demo syntax. Users making minute adjustments on a ggplot, for example, can easily issue very similar ggplot( commands a few dozen times thereby cluttering the script file with fluff.
4. Use R through a script editor
To get better use from .Rhistory it is useful to use R through an editor like EMACS (vis ESS) or VI. These editors bring highlighting, colour coding, spell checking, formatting and other facilities to both .Rhistory and the current session. More importantly syntax editors bring search functionality into the active session of R and .Rhistory. The use of a script editor makes it easy to find every instance where a particular command or string was used or a particular variable referred to. Clearly this can be done by simply searching the .Rhistory but with slight modifications it is possible to rotate through every instance a particular command was used. The user can, for example, rotate through every ggplot(), lm() or table() command to find when it was last used and what the syntax was.
Under EMACS a slight modification to the .emacs file in the home directory is needed. These changes enable users to type the first few letters of a command and and then use the up and down arrows to rotate through useage. The following text must be place in a personal .emacs file:
---------------------------------------------------------------------
(eval-after-load
"comint"
'(progn
(setq comint-input-ring-size 20480)
(setq comint-scroll-to-bottom-on-output 'others) ; not current
;;=default: (setq comint-scroll-to-bottom-on-input nil)
(setq comint-scroll-show-maximum-output t) ;;; this is the key
(define-key comint-mode-map [up]
'comint-previous-matching-input-from-input)
(define-key comint-mode-map [down]
'comint-next-matching-input-from-input)
(Define-key comint-mode-map "\C-a" 'comint-bol)))
----------------------------------------------------------------
Now, by typing "ggplot(" at the prompt and pressing the up arrow EMACS will show, in order, every command starting with the string "ggplot(" showing when it was used. While this is useful to iterate through your history its utility is enhanced by adopting standardised practices that allow easy access to key functions.
Instead of using the standard format of LHS <- eg.="" i="" rhs="">comment(dfname) <- i="" nbsp="">'this dataframe was altered by adding column called "imp_val"' )
routinely reverse the direction of assignment to: "xx -> yyy". For example the above command becomes ...
comment(dfname) -> 'this dataframe was altered by adding column called "imp_val" '
Now if, in EMACS, the user types "comment(" and press the up arrow EMACS will rotate through every comment and (hopefully) find the relevant assignment. Analysts are more likely to be looking for statistical procedures or plots rather than comment assignment. If every import of csv data followed the syntax of "read.csv(" followed by the assignment " -> importedfilename" the user can easily rotate through all the csv imports. It now becomes easy to examine processes used in creating the current database. If the direction of assignment was not reversed and remains "LHS <- -="" i="" rhs=""> then pressing the up arrow will not find commands but assignment objects. For this to be of value the user will need to know the assigned name.
5. Create a sparse model of core commands
The practice of following the above syntax format also has the advantage of enabling the user to extract every assignment with ease. Typically analysts will experiment with a procedure or plot until the desired results are achieved. Once the user is happy with the syntax the result/output is assigned a name (by using the " -> newname" syntax).
If this syntax format is routinely used then every change in a value or assignment can be extracted from .Rhistory with ease. Under linux this key syntax can be parsed to a new file (called "mychanges.txt") using a command like:
less .Rhistory | grep -E ' -> |%>%' > mychanges.txt
If the user has been inserting #comments into the syntax file then use:
less .Rhistory | grep -E ' -> |%>%|#' > mychanges.txt
The file "mychanges.txt' is a record of every assignment made in that database using either "->" or tidyverse's "%>%". This new file is a complete record of every change made to that database and contains all the information needed to replicate all the work to date.
Commands are often refined incrementally until the desired outcome is reached and the extracted "mychanges.txt" file will contain every iteration made. In practice analyst usually only want the final iteration to be preserved (analysis presumably stops when the solution has been identified). It is a little extra housekeeping to remove all iterations but the last in "mychanges.txt". Doing this should result in a sparse and clean syntax file that records all changes to the database and no 'fluff'.
6. Routinely use a file to record notes
The key to adequate documentation is being reminded to make notes and being able to do so easily. Database documentation can be improved by adding a decriptive file to record intentions, procedures, progress or conclusions to every R directory (and thus to every R database). To be consistent keep one of these files in every R directory and use the same name eg. "desc.txt".
On linux systems the process is facilitated by creating an alias for EMACS (or VI) that equates the intial command "emacs" with "emacs desc.txt". Now when the user navigates to the relevant R database/directory typing "emacs"results in "desc.txt" being created (or re-opened) and opened in EMACS. The user can thus edit this file while running R in another buffer ("Ctrl-x 2" to open a new buffer in which to run R via "Alt-x R").
On Debian systems the alias is set in the home directory's .bashrc or .bash_aliases file using a string like:
alias emacs="emacs -g -1 desc.txt" (or put this into a script as shown below)
The "desc.txt" file is used to record notes that don't quite belong in .Rhistory. Once the R session is over exiting requires that EMACS be closed. Before the user exits EMACS they will be prompted to save changes to "desc.txt". This can be done after adding notes that will guide future users. As the notes made gradually accumulate a useful addition to .Rhistory is created.
7. Routinely add a timestamp to .Rhistory
Placing a timestamp in .Rhistory may help you to find previously used procedures or identify when modifications were made to a database. However placing a timestamp in .Rhistory is not straight-forward as this file is only updated when you exit R. Also .Rhistory records only the commands issued and not the output (the date etc.). One workaround is to create a script that appends a timestamp to .Rhistory immediately on starting R. I combine the insertion of the timestamp with the opening of "desc.txt" referred to above through a script.
To use the script navigate to the working directory and start "emacsR.sh". emacsR.sh is a text file which contains the following text
8. Refer to R objects in external products
The best record of how a database was used/applied is kept by writing reports in R Markdown, or knitr. This is because the code 'chunks' in the scripts refers to the R objects directly. Users, for example, can see that a particular table was used by crosstbulating variables x and y and what the syntax was used in creating a particular plot.
Unfortunately many clients insist on tables, graphics etc. being placed in a word processor document (eg. Word or LibreOffice) document. By doing this they can apply their own styles easily or not have to extend their IT skills. If you are required to do this rather than pasting plots and tables into their document try linking to the R object (OLE?) instead. This way you may be able to preserve references to both the name of the R object used and the database/directory it camefrom. These references help when trying to work out what the author (you?) did last time.
9. Full circle
The use of many small databases coupled to the creation of sparse histories and descriptive files may result in a plethora of files and greater confusion. Fortunately on linux systems it is easy enough to examine every .Rhistory (or 'desc.txt' file) for a particular search phrase. Once the directory has been identified the user has a reference to the relevant database and is better able to locate the key information on procedures or data.
An example of a process may run something like this:
A user needs to change a plot they provided in a report last year.
1. From item 8. above the report In LibreOffice/Word contains a table produced by R called, say, "fiscalcliff".
2. Search every R database on the system that produced an object called "fiscalcliff". This is done by typing, in linux,
find / -name .Rhistory -exec grep -H "fiscalcliff" {} \;
The '-H' command above outputs the filename of relevance. Now you should know which directory is most relevant.
3. Navigate to the database and open 'mychanges.txt' created from step 5 above (or just use .Rhistory). Search (based on step 4 above) from the end of the file backwards for 'fiscalcliff' ("Ctrl-r fiscalcliff" in EMACS) to show the last time that term was used.
4. The mychanges.txt file will contain all the commands needed to replicate everything to get to that point of output. Theoretically the user merely has to paste all the relevant text into R to get to that point again.
Caveats
There are procedures that may not be saved using the above text varbatim. Both base plots and ggplot, for example do not output graphics using the ' -> ' or ' <- assignment.="" base="" i="" like="" plots="" something="" use="">png('filename.png')
and ggplot uses ggsave('filename.png'). These commands can be outputted to the sparse history (mychanges.txt) file by widening the grep term in:
less .Rhistory | grep -E ' -> |%>%|#' mychanges.txt
to include the additional terms. For example ggplot users may want to use:
less .Rhistory | grep -E ' -> |%>%|#|ggsave' > mychanges.txt
As someone who has to return to old R projects the management of 'legacy' databases is an on-going 'challenge'. Some analysis are frequently called on to replicated or modify previous work or to use old databases for new analysis. Knowing what is in a database, what changes have been made to it and how to repeat key processes is central to reusing those resources efficiently. Comprehensive documentation is obviously key to redeploying the databases efficiently but, candidly, documentation tends to be poor. Usually impatience coupled and the perception that analysis and procedures seemed perfectly clear, if not intuitive, at the time contributes to inadequate documentation. In practice returning to old databases usually results in considerable time being spent on deciphering .Rhistory scripts and divining what had been done previously. This invariably results in a few 'WTF' moments or total bewilderment when trying to reverse engineer what the earlier analyst did.
Fortunately a number of protocals and procedures can radically improve the maintenance of databases and scripts.
1. Keep databases (i.e. everything in the current R directory) small
By keeping databases small and focussed on key subjects it is easier to increase the level of documentation (see below) and eliminate the fluff that inevitably clutters working directories.
Ideally issuing the ls() command should result in as sparse an output as possible. Using multiple small databases allows for information to be compartmentalised (and thus 'filed') it may results in confusion as to which database should be used in the first place. As far as possible start every project in its own directory.
2. Issue all commands from the CLI.
To record all changes and procedures in scripts only issue commands through the command line (LI), this way a record of changes and procedures are always at hand. If you exit R using q('yes') R records all command line instructions in the hidden ".Rhistory" file. Unless you make extensive use of drop-down menus and some spreadsheet-like facilities, this automatically creates a record of the commands used. By systematically recording commands database changes and procedures become replicable.
Many spreadsheet-type actions are not recorded and using commands like fix(dataframename) results in changes that are not documented. Accordingly these routines should be avoided and all changes are made by issuing a command on the CLI.
Be generous in setting the memory allocated to ".Rhistory". On linux systems this is set using a command in a system-wide or personal .Rprofile file. This file should contain a line like:
Sys.setenv(R_HISTSIZE='100000')
The size should be large enough to ensure all changes to that database are recorded. As .Rhistory is a record of all commands submitted keeping these files clean requires regular housekeeping. Ideally .Rhistory should have only the commands required to replicate the output ab initio. The .Rhistory file can easily be cluttered by repeated iterations, experimentation or the running of demo syntax. Users making minute adjustments on a ggplot, for example, can easily issue very similar ggplot( commands a few dozen times thereby cluttering the script file with fluff.
3. Use "comments" everywhere you can.
"Comments" enable users to keep track of observations they wish to have a record of. and using them often is key to keeping track of modifications. Comments can be pegged to R objects like dataframes and dataframe columns using syntax like:
'Dataframe altered on 3 May 2016 by adding the column "imp_val"' -> comment(dfname)
or, referencing a column in a dataframe,
'These are imputed values for the age variable assuming missing obs were C.A.R.' -> comment(dfname$imp_val)
To retrieve the comment assigned to a particular dataframe column use:
comment(dfname$imp_val)
The main problem with comments is that they are concealed and users have to tell R to show the comment associated with an object. In other words the user must know, or suspect, that a comment was assigned to the relevant object. If users do not routinely use comment assignment (and retrieval) the facility easily falls into disuse.
"Comments" enable users to keep track of observations they wish to have a record of. and using them often is key to keeping track of modifications. Comments can be pegged to R objects like dataframes and dataframe columns using syntax like:
'Dataframe altered on 3 May 2016 by adding the column "imp_val"' -> comment(dfname)
or, referencing a column in a dataframe,
'These are imputed values for the age variable assuming missing obs were C.A.R.' -> comment(dfname$imp_val)
To retrieve the comment assigned to a particular dataframe column use:
comment(dfname$imp_val)
The main problem with comments is that they are concealed and users have to tell R to show the comment associated with an object. In other words the user must know, or suspect, that a comment was assigned to the relevant object. If users do not routinely use comment assignment (and retrieval) the facility easily falls into disuse.
4. Use R through a script editor
To get better use from .Rhistory it is useful to use R through an editor like EMACS (vis ESS) or VI. These editors bring highlighting, colour coding, spell checking, formatting and other facilities to both .Rhistory and the current session. More importantly syntax editors bring search functionality into the active session of R and .Rhistory. The use of a script editor makes it easy to find every instance where a particular command or string was used or a particular variable referred to. Clearly this can be done by simply searching the .Rhistory but with slight modifications it is possible to rotate through every instance a particular command was used. The user can, for example, rotate through every ggplot(), lm() or table() command to find when it was last used and what the syntax was.
Under EMACS a slight modification to the .emacs file in the home directory is needed. These changes enable users to type the first few letters of a command and and then use the up and down arrows to rotate through useage. The following text must be place in a personal .emacs file:
---------------------------------------------------------------------
(eval-after-load
"comint"
'(progn
(setq comint-input-ring-size 20480)
(setq comint-scroll-to-bottom-on-output 'others) ; not current
;;=default: (setq comint-scroll-to-bottom-on-input nil)
(setq comint-scroll-show-maximum-output t) ;;; this is the key
(define-key comint-mode-map [up]
'comint-previous-matching-input-from-input)
(define-key comint-mode-map [down]
'comint-next-matching-input-from-input)
(Define-key comint-mode-map "\C-a" 'comint-bol)))
----------------------------------------------------------------
Now, by typing "ggplot(" at the prompt and pressing the up arrow EMACS will show, in order, every command starting with the string "ggplot(" showing when it was used. While this is useful to iterate through your history its utility is enhanced by adopting standardised practices that allow easy access to key functions.
Instead of using the standard format of LHS <- eg.="" i="" rhs="">comment(dfname) <- i="" nbsp="">'this dataframe was altered by adding column called "imp_val"' )
routinely reverse the direction of assignment to: "xx -> yyy". For example the above command becomes ...
comment(dfname) -> 'this dataframe was altered by adding column called "imp_val" '
Now if, in EMACS, the user types "comment(" and press the up arrow EMACS will rotate through every comment and (hopefully) find the relevant assignment. Analysts are more likely to be looking for statistical procedures or plots rather than comment assignment. If every import of csv data followed the syntax of "read.csv(" followed by the assignment " -> importedfilename" the user can easily rotate through all the csv imports. It now becomes easy to examine processes used in creating the current database. If the direction of assignment was not reversed and remains "LHS <- -="" i="" rhs=""> then pressing the up arrow will not find commands but assignment objects. For this to be of value the user will need to know the assigned name.
5. Create a sparse model of core commands
The practice of following the above syntax format also has the advantage of enabling the user to extract every assignment with ease. Typically analysts will experiment with a procedure or plot until the desired results are achieved. Once the user is happy with the syntax the result/output is assigned a name (by using the " -> newname" syntax).
If this syntax format is routinely used then every change in a value or assignment can be extracted from .Rhistory with ease. Under linux this key syntax can be parsed to a new file (called "mychanges.txt") using a command like:
less .Rhistory | grep -E ' -> |%>%' > mychanges.txt
If the user has been inserting #comments into the syntax file then use:
less .Rhistory | grep -E ' -> |%>%|#' > mychanges.txt
The file "mychanges.txt' is a record of every assignment made in that database using either "->" or tidyverse's "%>%". This new file is a complete record of every change made to that database and contains all the information needed to replicate all the work to date.
Commands are often refined incrementally until the desired outcome is reached and the extracted "mychanges.txt" file will contain every iteration made. In practice analyst usually only want the final iteration to be preserved (analysis presumably stops when the solution has been identified). It is a little extra housekeeping to remove all iterations but the last in "mychanges.txt". Doing this should result in a sparse and clean syntax file that records all changes to the database and no 'fluff'.
6. Routinely use a file to record notes
The key to adequate documentation is being reminded to make notes and being able to do so easily. Database documentation can be improved by adding a decriptive file to record intentions, procedures, progress or conclusions to every R directory (and thus to every R database). To be consistent keep one of these files in every R directory and use the same name eg. "desc.txt".
On linux systems the process is facilitated by creating an alias for EMACS (or VI) that equates the intial command "emacs" with "emacs desc.txt". Now when the user navigates to the relevant R database/directory typing "emacs"results in "desc.txt" being created (or re-opened) and opened in EMACS. The user can thus edit this file while running R in another buffer ("Ctrl-x 2" to open a new buffer in which to run R via "Alt-x R").
On Debian systems the alias is set in the home directory's .bashrc or .bash_aliases file using a string like:
alias emacs="emacs -g -1 desc.txt" (or put this into a script as shown below)
The "desc.txt" file is used to record notes that don't quite belong in .Rhistory. Once the R session is over exiting requires that EMACS be closed. Before the user exits EMACS they will be prompted to save changes to "desc.txt". This can be done after adding notes that will guide future users. As the notes made gradually accumulate a useful addition to .Rhistory is created.
7. Routinely add a timestamp to .Rhistory
Placing a timestamp in .Rhistory may help you to find previously used procedures or identify when modifications were made to a database. However placing a timestamp in .Rhistory is not straight-forward as this file is only updated when you exit R. Also .Rhistory records only the commands issued and not the output (the date etc.). One workaround is to create a script that appends a timestamp to .Rhistory immediately on starting R. I combine the insertion of the timestamp with the opening of "desc.txt" referred to above through a script.
To use the script navigate to the working directory and start "emacsR.sh". emacsR.sh is a text file which contains the following text
sed -i -e '$a\ ' .Rhistory
echo 'Session started.... ' $(date +%Y-%m-%d:%H:%M) >> .Rhistory
sed -i -e '$a\ ' .Rhistory
emacs -g -1 desc.txt
Line 2 inserts a timestamp at the end of the .Rhistory file in the working directory. Lines 1 and 3 pad that text with a blank line and the last line opens "desc.txt" with emacs. Remember to make emacsR.sh executable ("sudo chmod a+x emacsR.sh").
8. Refer to R objects in external products
The best record of how a database was used/applied is kept by writing reports in R Markdown, or knitr. This is because the code 'chunks' in the scripts refers to the R objects directly. Users, for example, can see that a particular table was used by crosstbulating variables x and y and what the syntax was used in creating a particular plot.
Unfortunately many clients insist on tables, graphics etc. being placed in a word processor document (eg. Word or LibreOffice) document. By doing this they can apply their own styles easily or not have to extend their IT skills. If you are required to do this rather than pasting plots and tables into their document try linking to the R object (OLE?) instead. This way you may be able to preserve references to both the name of the R object used and the database/directory it camefrom. These references help when trying to work out what the author (you?) did last time.
9. Full circle
The use of many small databases coupled to the creation of sparse histories and descriptive files may result in a plethora of files and greater confusion. Fortunately on linux systems it is easy enough to examine every .Rhistory (or 'desc.txt' file) for a particular search phrase. Once the directory has been identified the user has a reference to the relevant database and is better able to locate the key information on procedures or data.
An example of a process may run something like this:
A user needs to change a plot they provided in a report last year.
1. From item 8. above the report In LibreOffice/Word contains a table produced by R called, say, "fiscalcliff".
2. Search every R database on the system that produced an object called "fiscalcliff". This is done by typing, in linux,
find / -name .Rhistory -exec grep -H "fiscalcliff" {} \;
The '-H' command above outputs the filename of relevance. Now you should know which directory is most relevant.
3. Navigate to the database and open 'mychanges.txt' created from step 5 above (or just use .Rhistory). Search (based on step 4 above) from the end of the file backwards for 'fiscalcliff' ("Ctrl-r fiscalcliff" in EMACS) to show the last time that term was used.
4. The mychanges.txt file will contain all the commands needed to replicate everything to get to that point of output. Theoretically the user merely has to paste all the relevant text into R to get to that point again.
Caveats
There are procedures that may not be saved using the above text varbatim. Both base plots and ggplot, for example do not output graphics using the ' -> ' or ' <- assignment.="" base="" i="" like="" plots="" something="" use="">png('filename.png')
and ggplot uses ggsave('filename.png'). These commands can be outputted to the sparse history (mychanges.txt) file by widening the grep term in:
less .Rhistory | grep -E ' -> |%>%|#' mychanges.txt
to include the additional terms. For example ggplot users may want to use:
less .Rhistory | grep -E ' -> |%>%|#|ggsave' > mychanges.txt
Monday, February 15, 2016
Projections for 2016 local government elections
Before
the promises are made, campaigning starts and voter–behaviour
surveys begin the only way to predict elections outcomes hold is to
examine the past behaviour of voters. The projection of past patterns gives insight
into what South Africans can expect when the local government
election takes place in the third quarter of 2016. While projections offers an indication as what to expect from the election it should be
borne in mind that the objective of election campaigns is largely to
alter the expected outcome.
A
clear signal of voter intentions for 2016 was already given by the
2014 National/ Provincial election. By extrapolating the changes in
voting patterns between the last local government election (2011) and
2014 we can predict that some changes are in the offing for 2016. In
particular the ANC can expect declines of about four percent support
in Gauteng, the North West province and the Eastern Cape. These
declines will be partly offset by a similar percentage gain in ANC
vote share in kwaZulu-Natal. The largest movements in votes will be
evident in the cities.
In
general the ANC can expect to receive a significantly reduced
proportion of the votes cast in metropolitan areas and several other
cities. As far back as the 2014 National/ Provincial elections the
ANC received less than half the proportional representation (PR)
votes cast in Nelson Mandela Bay. As that was not a municipal
election it did not make any difference to the composition of the
metropolitan council then. The changes in support profile are likely
to impact on the composition of the municipal council now.
The
forecasts indicate that the ANC, while retaining its minority vote
share in Cape Town, will lose the absolute majority it currently
holds in:
- Nelson Mandela Bay,
- Tshwane and
- Johannesburg.
In
these three metropoles the ANC will continue to be the single largest
political party but will need the support of opposition councillors
to set budgets, pass motions, and enact by-laws. After the election
the ANC will hold an absolute majority in only four of the eight
metropoles.
The
ANC is also likely to lose its absolute majority in other
municipalities which are highly urbanised. These include:
- Randfontein,
- Kouga,
- Thabazimbi and
- Rustenburg.
In
Rustenburg the ANC majority, which stood at over three-quarters of
votes cast before 2011, has since been whittled away by the
opposition – in particular by the EFF.
However
the election also portends important gains for the ANC particularly
in kwaZulu-Natal. Overall the ANC is set to gain absolute majorities
in another 28 municipalities. In almost all of the 28 municipalities
the ANC is currently the largest political party, albeit one with
less than 50% of popular support. After 2016 election the ANC may
well no longer need the support of the opposition to pass budgets and
by-laws. While most of the 28 municipalities are in kwaZulu-Natal a
handful are in the Western and Northern Cape.
In
six of the 28 municipalities the ANC stands to win the municipality
from another political party. These include winning (from the IFP):
- Nkandla and
- Mthonjeni.
Further
south the ANC seems to win several municipalities currently dominated
by the DA. These are:
- Bitou,
- Laingsburg,
- Hessequa and
- Witzenberg.
Exactly
how the changing vote patterns impact on the composition of elected
councils depends on both how the PR votes are apportioned between
wards and the second leg of the election (the election of ward
councillors directly). It seems that while a marginal change in the
political alignment of larger cities is indicated these changes may
have a deep impact on the quality of their governance. Until then
each of the political parties will strive to ensure that the outcomes
are better than what is expected now.
Detailed results are available at: Projections of ANC support in 2016 municipal elections.
Saturday, April 26, 2014
The real inflation rate
A perennial point of contention among South Africans is the disjuncture between the rate of inflation stated by officials and economists and their lived experience. Individuals increasingly suggest that the "real" rate of inflation is substantially higher than what, for example, StatsSA tells us it is. To support their claims individuals are usually able to cite innumerable examples as to why their experiences debunk the official CPI (currently at about 6%).
Economists, in retort, point that the CPI and other estimates of inflation are scientifically based and drawn from rigorously collected data and known consumption patterns A great deal of effort is invested by authorities in collecting prices for a predefined basket of goods that mirrors what is known about consumers consumption patterns and purchasing habits. The inflation rate is derived from objective empirical studies that cannot easily be discounted. Pundits then invariably proceed to point out that mismatches are possible as the consumption patterns are unique to individuals while the CPI measures the aggregate impact.
They further point out that critics of the official rate cite price increases that are most striking while they remain silent on those products whose prices are rising slowly or even falling. Cellphone charges and the internet bandwith costs have, for example, plummeted just as demand for these utilities become increasingly important.
However these responses indicate lack of insight - the (supposedly subjective) experience of the critics are not necessarily incompatible with the (objective) official statistics.
On many items consumers are confronted with price increases that significantly exceed the inflation rate. In recent months these items have come to include food and fuel. In addition administered prices (i.e. prices that are set by state institutions who do not face competition for their services) have also risen far faster than the CPI. Included among the latter are municipal rates, the price of electricity and road tolls. One feature common to these products is that there are few or no substitutes and consumers can do relatively little to circumvent them. To ensure there is food on the table, they are able to get to work and their municipal rates are paid individuals have to bear with these increases and cut costs elsewhere. The only place where costs can be cut are on discretionary items like IT and cellphone costs. It is among the latter category that price increases have been most modest.
In other words consumers are unable to take full advantage of stagnant or declining prices as their income is absorbed by the rapidly increasing cost of essentials and administered prices. In the process their cost profile rises and their consumption patterns alter. The basket of goods that they consume changes with prices with an ever increasing proportion of the basket being made up of high inflation essentials.
In the interim the economists are deriving the CPI on a stagnant basket of goods - not one where the 20% increase in the cost of food and petrol and electricity is offset by commensurate reduction in consumption elsewhere. The composition of the basket of goods used in deriving the CPI is only reconsidered ever ten years or so. In reality households have to adjust their consumption patterns every time there are sustained increases in prices and not every ten years or so.
So, dear consumer, you may not be losing your grasp on reality. Your personal rate of inflation may be well into the double digits and it is cold comfort that the price of items you can no longer afford are rising less rapidly. On the other hand I suggest that the official statistics will have ever less relevance to individual consumers.
Economists, in retort, point that the CPI and other estimates of inflation are scientifically based and drawn from rigorously collected data and known consumption patterns A great deal of effort is invested by authorities in collecting prices for a predefined basket of goods that mirrors what is known about consumers consumption patterns and purchasing habits. The inflation rate is derived from objective empirical studies that cannot easily be discounted. Pundits then invariably proceed to point out that mismatches are possible as the consumption patterns are unique to individuals while the CPI measures the aggregate impact.
They further point out that critics of the official rate cite price increases that are most striking while they remain silent on those products whose prices are rising slowly or even falling. Cellphone charges and the internet bandwith costs have, for example, plummeted just as demand for these utilities become increasingly important.
However these responses indicate lack of insight - the (supposedly subjective) experience of the critics are not necessarily incompatible with the (objective) official statistics.
On many items consumers are confronted with price increases that significantly exceed the inflation rate. In recent months these items have come to include food and fuel. In addition administered prices (i.e. prices that are set by state institutions who do not face competition for their services) have also risen far faster than the CPI. Included among the latter are municipal rates, the price of electricity and road tolls. One feature common to these products is that there are few or no substitutes and consumers can do relatively little to circumvent them. To ensure there is food on the table, they are able to get to work and their municipal rates are paid individuals have to bear with these increases and cut costs elsewhere. The only place where costs can be cut are on discretionary items like IT and cellphone costs. It is among the latter category that price increases have been most modest.
In other words consumers are unable to take full advantage of stagnant or declining prices as their income is absorbed by the rapidly increasing cost of essentials and administered prices. In the process their cost profile rises and their consumption patterns alter. The basket of goods that they consume changes with prices with an ever increasing proportion of the basket being made up of high inflation essentials.
In the interim the economists are deriving the CPI on a stagnant basket of goods - not one where the 20% increase in the cost of food and petrol and electricity is offset by commensurate reduction in consumption elsewhere. The composition of the basket of goods used in deriving the CPI is only reconsidered ever ten years or so. In reality households have to adjust their consumption patterns every time there are sustained increases in prices and not every ten years or so.
So, dear consumer, you may not be losing your grasp on reality. Your personal rate of inflation may be well into the double digits and it is cold comfort that the price of items you can no longer afford are rising less rapidly. On the other hand I suggest that the official statistics will have ever less relevance to individual consumers.
Friday, December 20, 2013
Quality of governance and voting
AG reports and governance
Each year every state
department is measured in terms of the extent they have adhered to
the laws and regulations laid down with respect to accounting
procedures, financial practice, asset control, awarding of contracts
and so on. These assessments by the Auditor General (A) speak to the
extent to which prescribed minimum standards of governance are met as
they present a concise assessment of, inter alia, the levels
of wastage and corruption in state departments. As they deliver a
seemingly ``objective'' opinion of government performance and are an
invaluable measurement of the potential effectiveness of state
agencies. The AGs reports on municipal government are of particular
interest as they indicate why service levels targets may not have
been met, budgets went unspent or how much money was misappropriated
or wasted.
Municipalities obtain
"clean" audits by being able to demonstrate that they have
systematically adhered to the regulations and legislation that govern
their functioning. Being able to demonstrate this depends on several
factors including the competence of accounting officers, the adequacy
of IT and HR systems and presence of effective disincentives for
financial abuse. Although service delivery depends heavily of local
government performance it is at this tier of government that these
factors tend to be least adequate. The MLGI Audit Barometer, for
example, shows that 60 percent of local municipalities are
substantially in breach of the prescribed regulations and
legislation. Moreover, the more optimistic projections indicate that
at the current rate of improvement it will be more than a decade
before all local municipalities receive "unqualified"
audits i.e. audits where substantial deviations from required
practice cannot be identified.
Poor audit outcomes are
often explained in terms of the lack of suitably qualified staff. The
appointment of staff with the required technical competence and the
ability to ensure conformance with prescribed policies is essential
to achieving a positive audit. It would seem, prima facie, that large
towns and cities would be best positioned to attract qualified staff.
While this is manifest by most metropolitan municipalities achieving
good audit outcomes the view is undermined by the fact that, of all
the provinces, the largely rural KwaZulu-Natal local municipalities
achieves the best audit outcomes. The MLGI Audit Barometer shows that
83 percent of local municipalities in that province routinely obtain
unqualified audits. KwaZulu-Natal is closely followed by the Western
Cape where 80 percent of municipalities get clean audits. By contrast
in highly urbanised Gauteng 40% of the local and metropolitan
municipalities do not routinely get unqualified audits. Similarly
many remote rural rural municipalities like uMzimkhulu and Impendle
obtain excellent audit outcomes. Obviously the inability to attract
suitable personnel is not a key determinant of audit outcomes and
reasons for the massive variations in audit outcome thus have to be
sought elsewhere.
Voting patterns
In democratic electoral
systems determining who fills key administrative positions depends on
how much sway political parties hold in the constituency.
Consequently elections play a pivotal role in determining who fills
key position and, in turn, the quality of service delivery and the
diligence with which the rules and regulations are adhered to. In the
last local government election in 2011 the ruling party gained
controlled of over 85 percent of the 234 local municipalities. Given
the predominance of the ANC and the massive variation in the local
government audit outcomes no obvious trend between voting patterns
and audit outcomes is apparent.
An analysis of voting
returns and audit outcomes for 2011/12 reveals that municipalities
led by opposition parties (the DA, IFP, NFP etc.) received notably
better audit outcomes than ANC led municipalities. On the face of
municipalities in the opposition-controlled Western Cape are more
likely to receive a positive audit than municipalities in other
provinces. However, as indicated above, the best audit outcomes are
in kwaZulu-Natal where 80 percent of the local municipalities are
controlled by the ruling party. In that province there is little
reason why IFP or NFP dominated municipalities should have better
audit outcomes than ANC led municipalities. None of these
municipalities boast the qualities that would predispose them to
positive audit outcomes any more than ANC controlled municipalities
in kwaZulu-Natal.
While opposition led
municipalities do obtain better audit outcomes this does not seem to
be the result of an opposition party being elected. Good audit
outcomes are apparent in as many municipalities led by the ruling
party. What a correlation between audit outcomes and voting returns
does show is that any municipality dominated by a single political
party predisposes it to poor audit outcomes. Conversely, not being
dominated by a single political party systematically improves audit
outcomes. The graphic below illustrates this. It contrasts the
distribution of votes obtained by the wining party in
- municipalities that received an unqualified audit with
- those that received a qualified (or worse) audit.
It is clear that in
those municipalities with qualified audits the winning party
typically received about 75% of the total vote. In the vast majority
of "qualified" audit municipalities the winning party
received at least two-thirds of all votes cast in the 2011 local
government elections.
By contrast the winning
party in municipalities who got unqualified audits routinely received
only about 55% percent of the votes cast. The area where coalitions
are required as no one party gained an absolute majority are shown by
the lightly shaded area. This shows that only a small percentage of
municipalities run by coalitions receive qualified audits. It is
seemingly strong political dominance by a single party at the polls
that predispose the municipality to poor audit outcomes.
The general trend is:
- If a single party wins 90 percent of the vote that municipality has virtually no chance of getting an unqualified audit.
- If two-thirds of the population votes for one party then the municipality has less than a one-in-three chance of getting an unqualified audit.
This later situation
approximates the current national situation where the ANC has 66% of
the vote share and 30% of municipalities get unqualified audits. If
the winning party in a municipality was to get a simple majority (50%
of the vote) then the municipality has twice as much chance of
getting an unqualified audit.
The kwaZulu-Natal
experience in particular shows that audit quality is less influenced
by which political party is elected and more by how much competition
that political party faces. Moreover this trend bears out
irrespective of which political party is dominant. This
interpretation indicates that if the ruling party maintained control
over the overwhelming majority of municipalities (say the current
85%) audit quality would be improved if its share of votes declined
in those municipalities. The same goes for the opposition parties
that win municipalities (but no opposition party has a share of votes
that remotely approximates the ANC national share of votes).
This relationship
indicates that the voters have much to do with audit outcomes and
thus directly influence governance quality. However the result has
less to do with changing which political party runs the municipality
and more to do with ensuring that there is a healthy counter to
political dominance. Having a strong opposition ensures greater
scrutiny of activities and policies, improved transparency and,
ultimately, better quality of governance. Voters can thus improve
audit outcomes (and governance quality) even by voting for parties
that have no chance of winning the election.
There has been a
tendency for disaffected voters in South Africa to abstain from
voting. However this results only is a race to the bottom as party
performance in elections then depends on who loses the least number
of supporters. This analysis shows that voters can improve governance
quality by transferring their vote to another party - even one
destined to lose. The one key question that has yet to answered is
whether or not voters do in fact reward performance at the polls.
Wednesday, December 5, 2012
Zuma and the rise of the right.
At the last general election the ANC, recently placed under the stewardship of Jacob Zuma, was returned to power with the mandate of another 2 million voters. A
t the time pundits attributed the increased support to trade union and the ANC Youth League support for Zuma. However since that election there has been a funda
mental realignment within the ANC which should become readily apparent in Mangaung next month. Those factions attributed with giving the ANC its increased mandate are now among those most opposed to Zuma being elected for a second term. By contrast those supporting Zuma are a strange mixture of "radicals" and conservatives including MK veterans, religious leaders, the SACP and traditional leaders.
The realignment does not represent Zumas betrayal of his backers. Rather it reflects the recognition that the additional votes came primarily from "tribal" areas. At last count approximately one-quarter of South Africans of voting age resided in what are now referred to as "traditional" areas. This makes this constituency one of the most influential in the country and in the ANC.
The features that unite Zuma supporters are becoming ever more apparent. Not only are these groups socially conservative, they tend to be highly patriarchal and, at heart, anti-democratic. To gain the support of these groups candidates have to focus on beguiling their leaders rather than the rank and file. Zuma has to appeal not to the soldiers, congregants and subjects but to an elite of generals, bishops and kings. Obviously this calls for a fundamentally different approach to the populism of the Youth League we grew accustomed to.
Zuma's pursuit of support from traditional leaders is shown by the ANC's dogged pursuit of "traditional courts". For some time the Department of Justice has been attempting to pass legislation which grants tribal chiefs the right to try criminal and civil cases and impose sanctions as they see appropriate. The procedures and sanctions adopted in these courts are to accord with traditional law and cultural practices.
The Traditional Courts Bill has stalled on repeated rejection by the affected communities. While the media has tended to emphasise the criticism voiced by women, opposition to the bill is widespread. In the public hearings on the bill men have tended to dominate proceedings both numerically and in vociferousness. They dismiss the bill with as much vigour as that attributed to women. Despite the opposition (which extends into cabinet) the Zuma administration continues to promote the legislation.
Despite the opposition and its obvious unconstitutionality the bill being pushed through because of the advantages it offers traditional leaders. In return traditional leaders can deliver the support of their subjects - at both Mangaung and at the ballot box.
The ANC will probably deny that the phalanx of legislation focussing on traditional areas entrenches partriarchal and anti-egalitarian systems. Admittedly there are now more women on Traditional Councils. Legislation requires that 30 percent of representatives are women. This is all well and good except for the fact that there is no popular support for Traditional Councils and these Councils have no power to speak of. Not only do these elections not meet the criteria for being free-and-fair turnout in Traditional Council elections typically run at less than ten percent of the eligible population. The reason for the low turnout lies in the negligible influence elected representatives have in the Councils. Most people on the council are not elected but are appointed by the tribal leaders. Moreover, just as traditional courts are not compelled to observe the precepts of natural justice, the Traditional Councils are not required to adhere to the basic principles of democratic practice. Chiefs are not bound by the majority vote of the Councils. Given this it is surprising that so many people bother to vote in Traditional Council elections at all.
Constitutional democrats may well have felt relief when the populist Youth Leagues fell from Zuma's grace. Current tension between the ANC and unions may even comfort those who fear the influence of the left within the ANC. However the alignment of the ANC leader with anti-democratic factions will prove to be far more worrisome. When he resebtly addressed the National House of Traditional Leaders Zuma encouraged them to reject "white mans justice". His justification rests on the view that "white mans justice" turns on cold facts rather than the "warm bodies" prioritised by indigenous justice systems. Zuma echoed criticisms of the judicial system as retributive, overly codified and "un-african". However the constitution now forms the bedrock of that "white mans justice". This perspective coupled with the bulldozing of the Traditional Courts and entrenching the undemocractic Traditional Councils on the most vulnerable communities in the country suggest that Zuma may be similarly predisposed to dismissing the constitution. He seems ready to court traditional authorities in the name of "Africanness" regardless of how it undermines democracy or how devisive it is within the ANC.
t the time pundits attributed the increased support to trade union and the ANC Youth League support for Zuma. However since that election there has been a funda
mental realignment within the ANC which should become readily apparent in Mangaung next month. Those factions attributed with giving the ANC its increased mandate are now among those most opposed to Zuma being elected for a second term. By contrast those supporting Zuma are a strange mixture of "radicals" and conservatives including MK veterans, religious leaders, the SACP and traditional leaders.
The realignment does not represent Zumas betrayal of his backers. Rather it reflects the recognition that the additional votes came primarily from "tribal" areas. At last count approximately one-quarter of South Africans of voting age resided in what are now referred to as "traditional" areas. This makes this constituency one of the most influential in the country and in the ANC.
The features that unite Zuma supporters are becoming ever more apparent. Not only are these groups socially conservative, they tend to be highly patriarchal and, at heart, anti-democratic. To gain the support of these groups candidates have to focus on beguiling their leaders rather than the rank and file. Zuma has to appeal not to the soldiers, congregants and subjects but to an elite of generals, bishops and kings. Obviously this calls for a fundamentally different approach to the populism of the Youth League we grew accustomed to.
Zuma's pursuit of support from traditional leaders is shown by the ANC's dogged pursuit of "traditional courts". For some time the Department of Justice has been attempting to pass legislation which grants tribal chiefs the right to try criminal and civil cases and impose sanctions as they see appropriate. The procedures and sanctions adopted in these courts are to accord with traditional law and cultural practices.
The Traditional Courts Bill has stalled on repeated rejection by the affected communities. While the media has tended to emphasise the criticism voiced by women, opposition to the bill is widespread. In the public hearings on the bill men have tended to dominate proceedings both numerically and in vociferousness. They dismiss the bill with as much vigour as that attributed to women. Despite the opposition (which extends into cabinet) the Zuma administration continues to promote the legislation.
Despite the opposition and its obvious unconstitutionality the bill being pushed through because of the advantages it offers traditional leaders. In return traditional leaders can deliver the support of their subjects - at both Mangaung and at the ballot box.
The ANC will probably deny that the phalanx of legislation focussing on traditional areas entrenches partriarchal and anti-egalitarian systems. Admittedly there are now more women on Traditional Councils. Legislation requires that 30 percent of representatives are women. This is all well and good except for the fact that there is no popular support for Traditional Councils and these Councils have no power to speak of. Not only do these elections not meet the criteria for being free-and-fair turnout in Traditional Council elections typically run at less than ten percent of the eligible population. The reason for the low turnout lies in the negligible influence elected representatives have in the Councils. Most people on the council are not elected but are appointed by the tribal leaders. Moreover, just as traditional courts are not compelled to observe the precepts of natural justice, the Traditional Councils are not required to adhere to the basic principles of democratic practice. Chiefs are not bound by the majority vote of the Councils. Given this it is surprising that so many people bother to vote in Traditional Council elections at all.
Constitutional democrats may well have felt relief when the populist Youth Leagues fell from Zuma's grace. Current tension between the ANC and unions may even comfort those who fear the influence of the left within the ANC. However the alignment of the ANC leader with anti-democratic factions will prove to be far more worrisome. When he resebtly addressed the National House of Traditional Leaders Zuma encouraged them to reject "white mans justice". His justification rests on the view that "white mans justice" turns on cold facts rather than the "warm bodies" prioritised by indigenous justice systems. Zuma echoed criticisms of the judicial system as retributive, overly codified and "un-african". However the constitution now forms the bedrock of that "white mans justice". This perspective coupled with the bulldozing of the Traditional Courts and entrenching the undemocractic Traditional Councils on the most vulnerable communities in the country suggest that Zuma may be similarly predisposed to dismissing the constitution. He seems ready to court traditional authorities in the name of "Africanness" regardless of how it undermines democracy or how devisive it is within the ANC.
Tuesday, November 6, 2012
========================================================
The results of the 2011 South African census were release in November 2011. The top-line results revealed a few surprises including a higher than expected population growth rate. The population total of 51.7 million was 2.5 percent greater than StatsSA's mid-year population estimates for that year. This indicates a population 1.2 million more than anticipated.
Apparently the census also indicates an increase in the Total Fertility Rate (TFR) which rose to 2.7 from the 2.5 of a decade earlier. Apparently an increase in TFR typified all the major population groups classified in the census. While an eight percent increase in TFR may not appear unduly worrisome it actually has has profound implications for our understanding of development. In terms of Demographic Transition Theory an increase in TFR (and by implication in the IMR) indicates a society under great stress. Recent examples of such increase include war-time Afghanistan and Zambia under the IMF structural adjustment programme. The IMF's structural adjustment programmes greatly restricted access to health services in Zambia allowing for massive increased in mortality and fertility rates.
According to StatsSA the past ten years in South Africa has been marked by just such trauma. Part of the trauma is related to the impact of HIV/AIDS on the dominant population group. HIV/AIDS may have resulted in the observed increase in mortality rates. In response families were prompted to having a greater number of children giving rise to higher TFR.
However the data revealed by StatsSA shows that the increase in family size has primarily taken place in the past past five years. The graphic below shows the marked increase in the youngest five year age cohort.
The last five years has coincided with the period in which the impact of HIV/AIDS was greatly reduced by widespread uptake of AVRs. HIV/Aids thus offers an unconvincing argument for the increase in TFR.
The results of the census thus either confirm that a) South African society has been under such profound stress that the Demographic Transition has been reversed or b) a more mundane answer lies in the quality of StatsSA's tallying.
Monday, August 20, 2012
South African democracy: the beginning of the end?
An instinctive response to the killing of 34 strikers by the police on August 16 is that the event may herald the end of state tolerance of dissent. Along with the 34 the assumption that state violence will not be lightly used to suppress opposition died. Although much public debate on the reasons for the incident followed the incident these discourses centered on explanations for the miners militancy or rationales for the polices' violence. The first perspective tended to "justify" the strikers actions in terms of their personal circumstances or on rivalry between competing trade unions.
The miserable social conditions that allegedly underpins the strikers militancy broadly reflects what happens in areas where the local economy is collapsing. Declining service levels, increasing unemployment, income supplementation through prostitution, illicit brewing, alcohol sales, crime etc. now typify the areas surrounding many platinum mines like Lonmin. These difficult circumstances contribute to miners militancy and to the ability of a upstart union to organise industrial action under the nose of a trade union that has long dominated the industry. The arguments that focus on the impact of social conditions are undermined by the clear focus the miners had on remuneration and by the fact that the strikers are probably better paid than those who shot them.
While the social conditions may have contributed to discrediting the established National Union of Mineworkers (NUM) they do not offer insight into why the police played a central role in the rivalry.
Although NUM has long been established as the main union representing underground workers at Lonmin the strikes were organized by a fledgling rival AMCU. Clearly the dominance of the ANC-aligned National Union of Mineworkers (NUM) was challenged by the strike and some pundits viewed the heavy-handed police action as state support for their embattled political ally.
Much public debate focused on the militancy of the miners and the threat they posed to the police and other miners. It is repeatedly pointed out that by the time the strikers stormed off the hillock on which they had gathered ten people (including two police officers) had already been murdered in the conflict. It is from his vantage point that most discussions around the appropriateness of the police response proceeds. This discourse then revolved around assessments of the threat posed by the strikers, the ability of the police to protect themselves and others etc.
However a central ingredient is missed in all these debates. By the time the strikers were driven from the hillock by police teargas the order to use lethal force (sharp ammunition) had already been issued. When the strikers stormed off the hillock towards the police the SAPS had already dispensed with crowd control control formations and non-lethal gear. The police has already demobilised the public-order police units in favour of deploying the elite National Intervention Unit (NIU). The NIU is a unit tasked with dealing with high risk conflict situations. The shields, dogs, batons, perimeter control mechanisms that are central to containing violent situations had been withdrawn in favour of a large, highly trained force armed only with lethal weapons.
Furthermore the police were deployed in a pattern that ensured that the strikers could not but run towards them if they were to escape the teargas lobbed at them. When they stormed off the hillock they strikers could not do anything but run towards the police who, in turn, we're unable to respond with anything other than their automatic weapons primed with sharp rounds. And fire the police did - until all the strikers were down. The strikers were not given an opportunity to surrender or escape.
In other words points about police preparedness, their level of training, the threat the strikers posed etc. are moot - the die had been cast before the first volleys of teargas were lobbed at the hill.
Authority to use sharps rounds can only be issued by the ranking SAPS officer - in this situation this was the recently appointed commissioner Phiyega. The resultant massacre would, on the face of it, support calls for police commissioners to be appointed from within police ranks. Perhaps a commissioner more familiar with crowd control and the volatile situations may have prevented the biggest mass ace in South Africa since Sharpeville. By intimation it was the commissioners inexperience that caused the massacre. However this perspective is undermined by the explanations that followed the incident. The day's events were followed by a sanctimonious blathering about the danger the strikers posed to others and how they were belligerent and superstitious they were. The subtext of these explanations was that the strikers got what they deserved. The fact that there were no police casualties on the day of the massacre suggests that the magnitude of the threat posed by the strikers was probably exaggerated.
However an explanation is still needed as to why the NIU had been deployed and the non-lethal measures withdrawn. Was this, for example, an attempt to neutralise a militant rival to the established union? Such explanations miss a key dimension to the conflict - the close alignment between Lonmin and the upper echelons of the ruling party. For example in its former guise as "Lonrho", Lonmin reputedly bankrolled several of the leading ANC families both while they were in exile and on their return to South Africa in the mid 1990s. Currently ANC heavyweight Cryril Ramaphosa is a (non-executive) director of Lonmin. It was also Lonmin that stood to benefit most from the ending of the strike. The platinum sector is going through a crisis with falling PGM prices, greatly reduced demand for their product as well as rapidly increasing costs structure. Of particular importance is the cost of electricity used by the mines and associated processors. These factors ensure that not only could the mine not accede to the strikers demands they could not afford to cover running costs while a small component of the total workforce halted production. Only about ten percent of the Lonmin workforce joined the strike yet those 3 000 rock-drillers were able to stop production.
Clearly Lonmin stood to benefit most from the collapse of the strike than anybody. The events of the day represent nothing more sophisticated than the state acting in the interests of big capital and the ruling party. Unless the commission of inquiry is able to provide a more convincing account as to why non-lethal force was withdrawn we really do need to ask whether or not this is the end of the tolerance of dissent required of a functioning democracy.
The miserable social conditions that allegedly underpins the strikers militancy broadly reflects what happens in areas where the local economy is collapsing. Declining service levels, increasing unemployment, income supplementation through prostitution, illicit brewing, alcohol sales, crime etc. now typify the areas surrounding many platinum mines like Lonmin. These difficult circumstances contribute to miners militancy and to the ability of a upstart union to organise industrial action under the nose of a trade union that has long dominated the industry. The arguments that focus on the impact of social conditions are undermined by the clear focus the miners had on remuneration and by the fact that the strikers are probably better paid than those who shot them.
While the social conditions may have contributed to discrediting the established National Union of Mineworkers (NUM) they do not offer insight into why the police played a central role in the rivalry.
Although NUM has long been established as the main union representing underground workers at Lonmin the strikes were organized by a fledgling rival AMCU. Clearly the dominance of the ANC-aligned National Union of Mineworkers (NUM) was challenged by the strike and some pundits viewed the heavy-handed police action as state support for their embattled political ally.
Much public debate focused on the militancy of the miners and the threat they posed to the police and other miners. It is repeatedly pointed out that by the time the strikers stormed off the hillock on which they had gathered ten people (including two police officers) had already been murdered in the conflict. It is from his vantage point that most discussions around the appropriateness of the police response proceeds. This discourse then revolved around assessments of the threat posed by the strikers, the ability of the police to protect themselves and others etc.
However a central ingredient is missed in all these debates. By the time the strikers were driven from the hillock by police teargas the order to use lethal force (sharp ammunition) had already been issued. When the strikers stormed off the hillock towards the police the SAPS had already dispensed with crowd control control formations and non-lethal gear. The police has already demobilised the public-order police units in favour of deploying the elite National Intervention Unit (NIU). The NIU is a unit tasked with dealing with high risk conflict situations. The shields, dogs, batons, perimeter control mechanisms that are central to containing violent situations had been withdrawn in favour of a large, highly trained force armed only with lethal weapons.
Furthermore the police were deployed in a pattern that ensured that the strikers could not but run towards them if they were to escape the teargas lobbed at them. When they stormed off the hillock they strikers could not do anything but run towards the police who, in turn, we're unable to respond with anything other than their automatic weapons primed with sharp rounds. And fire the police did - until all the strikers were down. The strikers were not given an opportunity to surrender or escape.
In other words points about police preparedness, their level of training, the threat the strikers posed etc. are moot - the die had been cast before the first volleys of teargas were lobbed at the hill.
Authority to use sharps rounds can only be issued by the ranking SAPS officer - in this situation this was the recently appointed commissioner Phiyega. The resultant massacre would, on the face of it, support calls for police commissioners to be appointed from within police ranks. Perhaps a commissioner more familiar with crowd control and the volatile situations may have prevented the biggest mass ace in South Africa since Sharpeville. By intimation it was the commissioners inexperience that caused the massacre. However this perspective is undermined by the explanations that followed the incident. The day's events were followed by a sanctimonious blathering about the danger the strikers posed to others and how they were belligerent and superstitious they were. The subtext of these explanations was that the strikers got what they deserved. The fact that there were no police casualties on the day of the massacre suggests that the magnitude of the threat posed by the strikers was probably exaggerated.
However an explanation is still needed as to why the NIU had been deployed and the non-lethal measures withdrawn. Was this, for example, an attempt to neutralise a militant rival to the established union? Such explanations miss a key dimension to the conflict - the close alignment between Lonmin and the upper echelons of the ruling party. For example in its former guise as "Lonrho", Lonmin reputedly bankrolled several of the leading ANC families both while they were in exile and on their return to South Africa in the mid 1990s. Currently ANC heavyweight Cryril Ramaphosa is a (non-executive) director of Lonmin. It was also Lonmin that stood to benefit most from the ending of the strike. The platinum sector is going through a crisis with falling PGM prices, greatly reduced demand for their product as well as rapidly increasing costs structure. Of particular importance is the cost of electricity used by the mines and associated processors. These factors ensure that not only could the mine not accede to the strikers demands they could not afford to cover running costs while a small component of the total workforce halted production. Only about ten percent of the Lonmin workforce joined the strike yet those 3 000 rock-drillers were able to stop production.
Clearly Lonmin stood to benefit most from the collapse of the strike than anybody. The events of the day represent nothing more sophisticated than the state acting in the interests of big capital and the ruling party. Unless the commission of inquiry is able to provide a more convincing account as to why non-lethal force was withdrawn we really do need to ask whether or not this is the end of the tolerance of dissent required of a functioning democracy.
Tuesday, June 5, 2012
The ANC is the new IFP
Is the ANC the new IFP?
The National Chamber of Provinces recently concluded public hearing on the Traditional Courts Bill. This bill seeks to give traditional leaders the power to preside over certain civil disputes and criminal offences in the areas under their jurisdiction. The public hearings were an opportunity for those affected by the proposed legislation to voice their concerns about its impact. It was also an opportunity for the provincial legislatures to claim they had consulted affected communities before they endorsed the legislation.
Not surprisingly the bill was met with a great deal of resistance by civil society and community members. It is clear from the public hearings that community members are concerned that traditional leaders, some of whom are corrupt, ineffectual or partisan, are formally being given greater powers. Civil societies' objections centred on the negative impact the bill, if passed, will have on women's rights. The general thrust of criticism is that hereditary leadership in a highly patriarchal and authoritarian social setting undermines the rights of all commoners, particularly women, and is in clear conflict with the constitutions core premise of "equality". Essentially the bill entrenches the right,by birth, of some to judge and govern and the obligation of all others, by virtue of where they live, to be judged and ruled.
The hearings in kwaZulu-Natal were particularly enlightening. They showed unequivocal support among traditional leaders for the bill. The reason for this consensus is clear as senior traditional leaders are all set to benefit from this and other proposed changes. However the most remarkable part of the hearings was the absence of the antipathy between the ANC and the IFP that has long marked politics in kaZulu-Natal - particularly when traditional rights and culture were concerned. There was no sign of dissent between chiefs aligned to different parties or between the chiefs and the sponsors of the bill i.e. Luthuli House.
This consensus highlights the extent to which the political landscape in kwaZulu-Natal has changed. Those seeking to preserve traditional African culture, hereditary rights or general social conservatism in kwaZulu-Natal can now identify with the ruling party rather than with the IFP whose influence has been waning since 1994. The ANC has managed to appropriate a rich political terrain once monopolised by the IFP. Understanding how much the terrain has changed rests on acknowledgment of the extent to which the ANC has moved to the IFP's value system rather than through the political evolution of Zululand residents. It is this change in the ANC that invites the perspective that the ANC is the new IFP. It is the ANC that now promises to preserve hereditary rights, traditional values and promote the associated social conservatism.
The reason for the shift in the ANC is clear. In 2009 the ANC was returned to power with votes just shy of a two thirds majority. When compared to the previous national election (in 2004) voters gave the ANC an additional 1.2 million votes. This increase in votes was widely seen as an endorsement of the ANC under Zuma's leadership. However the addition votes were garnered not, as the political pundits claimed, from the left, the unions or from the youth. The increase in votes were almost entirely from traditional areas in kwaZulu-Natal and other provinces. As people living in other provinces on "traditional" land had largely thrown their lot in with the ANC already their contribution to the increased ANC vote was less pronounced.
Traditional leaders are key to the continued support of any political party in what was previously referred to as "tribal areas". The chiefs exert tremendous influence on whether or not people vote and on whom they vote for. For example in the 2009 election the IFP still managed to get 800 000 votes and these votes are now, in a very real sense, there for the taking. Given the changing political landscape there is little reason to suggest that the chiefs will take them to the new National Freedom Party or keep them with the IFP.
Inevitably there is a qui pro quo for this support. The first sign of the quid pro quo was the establishment of a Department of Rural Development and Land Reform within a month of the election. This was followed by the re-invigoration of traditional councils and , belatedly, the holding of traditional council elections. Reviving the Traditional Courts Bill (which had stalled in 2008) followed. Still to come is the National Traditional Affairs Bill which may well extend local government powers to traditional leaders.
All this bodes poorly for the IFP as traditional leaders no longer have to look to the IFP to advance their interests and traditional rights in general. The ANC is now happily fulfilling this role - even at the expense of the core values of the constitution and what the ANC's has been describing as its own political heritage. This said, the support that the traditional leaders bring will inevitably be at the cost of some of support it currently enjoys from what the ANC itself terms "progressives". For example, although the Womens League has been silent about the promotion of the Traditional Courts bill their complicity in the promotion of such patriarchal systems can not continue indefinitely. Until then the extent to which ANC support is derived from traditional leaders is the extent to which Zuma's position will be secure relative to the "progressives".
Thursday, April 26, 2012
South Africa and BRICsa - fading to nothing?
South Africa recently became the latest member of the BRIC (Brazil, Russia, Indian and China) group of economies. Membership of BRIC should carry, if anything, at least the promise of rapid economic growth. Unfortunately South Africa however has not been playing in the same league as the original members of BRIC in either terms of size or the rate of economic growth. This point can best be illustrated by contrasting South Africa and China (the biggest economy in BRICSA).
In 2011, according to Wikipedia http://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal) the Gross Domestic product (GDP) of China was US$ 7,298,147 million. By contrast the South African GDP was a "mere" US$ 408,074 million.
The major imbalance is highlighted by differential growth rates. In 2011 the growth rate of China may fall to as low as 8% per annum. South Africa, by contrast, would be fortunate to reach 4% growth. Using these figures it is easy to see that the Chinese economy will grow by US$ 583,851 million. This is 37 percent greater than the entire South African GDP - even when accounting for the 4% growth rate. In other words the growth of the Chinese economy alone exceeds, each year, the size of the South African economy. This discrepancy will be aggravated by the big differences in annual growth rates. These factors will ensure that, should current trends continue, South Africa will become an ever less significant part of BRICSA each year.
However being (relatively) small is not a measure of potential and, past growth is not an indication of what will happen in the future. What is an indication of potential are the policies and practices of the respective governments. China epitomises the "developmental state" a label South Africa statesmen adopt at every opportunity. The defining feature of the developmental state is its prioirtisation of economic growth. In other words these economies prioritize growth above other factors like government control and even, other things being equal, profitability.
It is here where South Africa hamstrings itself by prioritising the political agenda of the ruling party above economic growth - even while stating that economic development is its primary objective. In essence economic growth is subverted by a political agenda which seeks greater control of the economy by central government while refashioning the racial profile of the social and economic landscape - even as this agenda undermines service delivery and economic development.
In 2011, according to Wikipedia http://en.wikipedia.org/wiki/List_of_countries_by_GDP_(nominal) the Gross Domestic product (GDP) of China was US$ 7,298,147 million. By contrast the South African GDP was a "mere" US$ 408,074 million.
The major imbalance is highlighted by differential growth rates. In 2011 the growth rate of China may fall to as low as 8% per annum. South Africa, by contrast, would be fortunate to reach 4% growth. Using these figures it is easy to see that the Chinese economy will grow by US$ 583,851 million. This is 37 percent greater than the entire South African GDP - even when accounting for the 4% growth rate. In other words the growth of the Chinese economy alone exceeds, each year, the size of the South African economy. This discrepancy will be aggravated by the big differences in annual growth rates. These factors will ensure that, should current trends continue, South Africa will become an ever less significant part of BRICSA each year.
However being (relatively) small is not a measure of potential and, past growth is not an indication of what will happen in the future. What is an indication of potential are the policies and practices of the respective governments. China epitomises the "developmental state" a label South Africa statesmen adopt at every opportunity. The defining feature of the developmental state is its prioirtisation of economic growth. In other words these economies prioritize growth above other factors like government control and even, other things being equal, profitability.
It is here where South Africa hamstrings itself by prioritising the political agenda of the ruling party above economic growth - even while stating that economic development is its primary objective. In essence economic growth is subverted by a political agenda which seeks greater control of the economy by central government while refashioning the racial profile of the social and economic landscape - even as this agenda undermines service delivery and economic development.
Thursday, February 16, 2012
iPad and Vodacom pay-as-you go
The 3G connection on my iPad suddenly stopped working for no apparent reason. It kept telling me that I had no data left and prompted me to add data "now" or "later". This was despite me having just bought bundles of data.
I eventually tracked the problem to an invitation to update my settings (which I accepted). Essentially this change the APN setting under "Cellular Data" from "internet" to "iphone.vodacom.za".
Changing the setting back to"internet" solves the problem (leave the Username and Password blank).
It seems that Apple and/or Vodacom are keen to create the eronneous impression that only 3G subscription services, and not prepaid, works on the iPad.
I eventually tracked the problem to an invitation to update my settings (which I accepted). Essentially this change the APN setting under "Cellular Data" from "internet" to "iphone.vodacom.za".
Changing the setting back to"internet" solves the problem (leave the Username and Password blank).
It seems that Apple and/or Vodacom are keen to create the eronneous impression that only 3G subscription services, and not prepaid, works on the iPad.
Saturday, October 22, 2011
Huawei E367 on linux
The Huawei E367 modem is a fast USB modem for 3g connectivity. It worked out-of-the box when connected to Telkom's 8ta service via a TP-LINK router. However when I attempt to use the modem directly (i.e. not through the router) I got the following error when installing the module:
FATAL: module vendor=0x12d1 not found
Fortunately getting the E367 to work on a linux box takes only about five minutes of effort. It requires the installation of the "modeswitch" package, configuration of the "modeswitch data", installation of the module and the usual setting up of the connection.
Install the "modeswitch" package (using apt, yum etc.) . At this stage installing the package will not necessarily solve the problem as the details of the vendor and product have yet to be added to the configuration. As root edit the file:
'/etc/usb_modeswitch.conf" (Debian systems).
Append the following text to that file.
########################################################
# Huawei E367
EnableLogging=1
DefaultVendor= 0x12d1
DefaultProduct=0x1446
TargetVendor= 0x12d1
TargetProductList="1001,1406,140b,140c,1412,141b,14ac,1506"
CheckSuccess=20
MessageEndpoint= 0x01
MessageContent="55534243123456780000000000000011062000000100000000000000000000"
##############################################################
Now, with the modem attached, run (as root) the command
"usb_modeswitch"
You should not have another usbserial device running so unplug it or run the command "rmmod usbserial".
At this stage you need to install the E367 module, as root do a:
"modprobe usbserial vendor=0x12d1 product=0x1506"
(or change product code as reflected by the command "lsusb").
You may now configure the modem using NetworkManager or whatever other system you use. For 8ta this usually requires indicting only the dial number *99# and the APN as "internet".
For full instruction see:
http://www.zeroshell.net/eng/forum/viewtopic.php?t=2986
FATAL: module vendor=0x12d1 not found
Fortunately getting the E367 to work on a linux box takes only about five minutes of effort. It requires the installation of the "modeswitch" package, configuration of the "modeswitch data", installation of the module and the usual setting up of the connection.
Install the "modeswitch" package (using apt, yum etc.) . At this stage installing the package will not necessarily solve the problem as the details of the vendor and product have yet to be added to the configuration. As root edit the file:
'/etc/usb_modeswitch.conf" (Debian systems).
Append the following text to that file.
########################################################
# Huawei E367
EnableLogging=1
DefaultVendor= 0x12d1
DefaultProduct=0x1446
TargetVendor= 0x12d1
TargetProductList="1001,1406,140b,140c,1412,141b,14ac,1506"
CheckSuccess=20
MessageEndpoint= 0x01
MessageContent="55534243123456780000000000000011062000000100000000000000000000"
##############################################################
Now, with the modem attached, run (as root) the command
"usb_modeswitch"
You should not have another usbserial device running so unplug it or run the command "rmmod usbserial".
At this stage you need to install the E367 module, as root do a:
"modprobe usbserial vendor=0x12d1 product=0x1506"
(or change product code as reflected by the command "lsusb").
You may now configure the modem using NetworkManager or whatever other system you use. For 8ta this usually requires indicting only the dial number *99# and the APN as "internet".
For full instruction see:
http://www.zeroshell.net/eng/forum/viewtopic.php?t=2986
Monday, October 3, 2011
Tornadoes in South Africa
Recently I had to estimate the probability of a tornado damaging a school in South Africa. The issue was made all the more topical by the appearance of two destructive tornadoes on 2 October. The one occurred in Duduza in the east rand and the other half way across the country in Ficksburg. Was this a freak incident or are tornadoes a greater danger than we suppose? Drawing on a study by Milford and Goliger 1997 it would appear that approximately 15 destructive (greater than force F0) tornadoes occur each year on average.
The graphic below shows the Milford and Goliger's estimates of the likelihood a tornado occurs (in a given year) in any 1km square area of the country. The likelihood ranges from an effective 0 in the Western and Northern Cape, through one in 100 000 in the Eastern Cape and Free State (including Ficksburg). The likelihood increases to one in 10 000 in parts of Gauteng and the KwaZulu Natal midlands (including Duduza).
The vast majority of Tornadoes go unreported as they occur in the sparsely populated areas - like large tracts of the Eastern Cape and Free State. Given that only about 15 destructive tornadoes occur each year the occurrence of two tornadoes in populated areas on the same day would be highly unusual. However there is some doubt as to whether the two could be classified as F1 or higher. F0 tornadoes are unlikely to damage permanent structures. Usually the damage associated with F0 tornadoes are the result of peripheral damage - like tree branches falling onto houses. Unfortunately if a F0 tornado is able to topple an old tree or break branches off they are more than likely to be able to level shacks in Duduza or force poorly built structures to collapse.
The two incidents may thus say more about the quality of housing than they do about the prevalence of tornadoes.
The graphic below shows the Milford and Goliger's estimates of the likelihood a tornado occurs (in a given year) in any 1km square area of the country. The likelihood ranges from an effective 0 in the Western and Northern Cape, through one in 100 000 in the Eastern Cape and Free State (including Ficksburg). The likelihood increases to one in 10 000 in parts of Gauteng and the KwaZulu Natal midlands (including Duduza).
The vast majority of Tornadoes go unreported as they occur in the sparsely populated areas - like large tracts of the Eastern Cape and Free State. Given that only about 15 destructive tornadoes occur each year the occurrence of two tornadoes in populated areas on the same day would be highly unusual. However there is some doubt as to whether the two could be classified as F1 or higher. F0 tornadoes are unlikely to damage permanent structures. Usually the damage associated with F0 tornadoes are the result of peripheral damage - like tree branches falling onto houses. Unfortunately if a F0 tornado is able to topple an old tree or break branches off they are more than likely to be able to level shacks in Duduza or force poorly built structures to collapse.
The two incidents may thus say more about the quality of housing than they do about the prevalence of tornadoes.
Saturday, October 1, 2011
Service delivery protests: a new trend?
In early September two violent "service delivery" protests took place in Gauteng. The first was in Thembalihle informal settlement outside Lenasia, the second was in Tembisa on the east Rand. Although media coverage treated the two protests as similar, dubbing them "service protests", they have very different origins and profoundly different implications for the political future of the province.
Thembalihle
The Thembalihle protests were the latest resurgence of a dispute that started at least two decades ago. This is an informal housing settlement located adjacent to Lenasia. However the settlement is located on land unsuited for habitation or development. Much of the ground in the region is dolomitic and prone to developing sinkholes which pose an ongoing threat to life and property. However sinkholes occur infrequently and thus do not appear, at least to Thembalihle residents, to present an imminent danger. Subsequently when Thembalihle residents contrast their service levels and the level of investment in infrastructure to that of their middle class Lenasia neighbours they feel aggrieved. It is these grievance that are at the root case of the periodic protests.
The Johannesburg Metro Council, which is responsible for failing to prevent the repeated resettlement of the area now seeks, once again, to relocate the residents to geologically stable land about 10km away. As the the proposed resettlement area is far from shops, amenities, employment opportunities and the social connections offered by Lenasia few Thembalihle residents are happy to be resettled. Unfortunately the geological conditions preclude significant investment in the Thembalihle and it is improbable that grievances about service levels and investment levels will be addressed. The capital investment required to assuage grievances can well be considered as wasted investment or, even worse, a means of encouraging the settlement on unstable land.
Thembalihle residents are thus faced with the option of relocating or to continue occupying the unstable ground. If the council does make a serious attempt to relocate residents a breakdown in the relationships between council and residents may ensue. If residents wish to remain in the area in the face of such a resettlement drive they will possible make the area a no-go zone as far as the council is concerned.
However residents creating a no-go area for the council and, possibly, rendering the area ungovernable is not he worst case scenario being faced. The latter is represented by accelerated sinkhole activity. Although the informal settlement has been geographically stable for some time the decline of mining activity in the larger region has resulted in a rapidly rising water table. Moreover the water now entering the dolomitic deposits is highly acidic - it is the infamous "acid mine water" about which several apocalyptic predictions centre. These two factors (the rising water table and its acidity) promise a marked increase in sinkhole activity - and there is little doubt that Thembalihle residents now face increased risks.
The protests in Thembalihle represent one heritage of poor urban management as Johannesburg Metro Council has never demonstrated the political will to ensure that the area is not used for housing. Resettlements that took place in the past were all accompanied by an immediate resettlement of the land. The failure of adequate urban land management gives rise to the current impasse as grievances cannot be addressed (at least not safely).
Fortunately the disputes in Thembalihle are sure to remain localized, however the same can not be said of the protests at Tembisa.
Tembisa
Tembisa represents a dramatically different situation and has far more wide ranging implications. In Tembisa protests were sparked less by the absence of services than by the poor quality of the services and the cost of those services. Residents cited many grievances but most prominent among their grievances was the rapidly rising cost of electricity. This was blamed, variously, on "overcharging" by the council and the installation of pre-paid meters. The impact of electricity prices and suspicions about Ekhuruleni metro Councils efficacy was most vividly expressed by calls that ESKOM be made the service electricity provider.
This latter demand seems to have been particularly misguided. City councils merely resell electricity bought in bulk from ESKOM. The prices of many services provided by the state (electricity, gas, water, road tolls, license fees) are determined administratively i.e. they are determined by the interaction of supply and demand in a competitive setting but on consideration of other issues. The other considerations include the need to provide free basic services, redress of inequities in service levels, affordability and the need to recoup costs. In other words prices are set by political consideration rather than by markets. Be this as it may the costs of services still have to be recouped from either the fiscus or from other taxes or charges.
In recent years administrative charges have been rising more rapidly than market-related prices. In March this year Business Day gave the example:
"... (in the) last month administered prices rose twice as much as general services and three times as much as the average price of consumer goods".
As ESKOM seeks to fund massive new capital projects increases in the price of electricity has been particularly pronounced. These price increase of approximately 25% each year for four years in a row is not exclusively due to capitalisation costs. They have been prompted by the need to (temporarily at least) reduce electrify consumption and pay very lucrative salaries in the parastatal. Recent reports indicate that the average salary in ESKOM is well in excess of R500 000 per annum. This is more than ten times the value of South Africa' per capita GDP (which, in turn, is higher than average wages). ESKOM is not alone in presenting massive wage bills. In the three years leading to 2010 the total municipal wage bill rose by 54%. Over this period the total number of municipal employees rose by only 5% indicating that average remuneration in municipalities is rising rapidly . All these costs - capitalisation costs, wage bills etc. have to recouped from consumers - hence the pressure of administered prices. These factors ensure that the cost of municipal services like electricity will continue to increase at above the rate of inflation - irrespective of whether the services are provided by the council or another service provider (like ESKOM).
The Tembisa protests indicate that those setting administrative prices have misread the willingness of township residents to bear the ever increasing remuneration and other costs.
Ultimately this is a political misjudgement by the elected representatives and by those appointed to oversee the provision of essential services. If the political representatives have indeed misread the willingness of consumers to absorb ever-increasing costs, as the Tembisa protests imply, then the latest protests may just be the first of many against the price of services. Perhaps we have entered a phase where an increasing proportion of public protests will centre not on the failure to deliver services but on the costs of those services.
What unites the Tembisa and Themblihle protests is the underlying lack of faith residents have in the metropolitan councils. In Thembalihle residents distrust the argument about the impact of dolomite as they witness better services and infrastructure across the road in Lenasia. Moreover they have not recently been affected by sinkholes. Much of the anger in Tembisa has been brought on by the belief that the council is overcharging them for electricity (hence he call for ESKOM to take over). Ultimately the combination of misjudgement by those setting prices and reservations about the integrity of elected representatives suggest that the issue of public order policing will become even more prominent in the years to come.
Thembalihle
The Thembalihle protests were the latest resurgence of a dispute that started at least two decades ago. This is an informal housing settlement located adjacent to Lenasia. However the settlement is located on land unsuited for habitation or development. Much of the ground in the region is dolomitic and prone to developing sinkholes which pose an ongoing threat to life and property. However sinkholes occur infrequently and thus do not appear, at least to Thembalihle residents, to present an imminent danger. Subsequently when Thembalihle residents contrast their service levels and the level of investment in infrastructure to that of their middle class Lenasia neighbours they feel aggrieved. It is these grievance that are at the root case of the periodic protests.
The Johannesburg Metro Council, which is responsible for failing to prevent the repeated resettlement of the area now seeks, once again, to relocate the residents to geologically stable land about 10km away. As the the proposed resettlement area is far from shops, amenities, employment opportunities and the social connections offered by Lenasia few Thembalihle residents are happy to be resettled. Unfortunately the geological conditions preclude significant investment in the Thembalihle and it is improbable that grievances about service levels and investment levels will be addressed. The capital investment required to assuage grievances can well be considered as wasted investment or, even worse, a means of encouraging the settlement on unstable land.
Thembalihle residents are thus faced with the option of relocating or to continue occupying the unstable ground. If the council does make a serious attempt to relocate residents a breakdown in the relationships between council and residents may ensue. If residents wish to remain in the area in the face of such a resettlement drive they will possible make the area a no-go zone as far as the council is concerned.
However residents creating a no-go area for the council and, possibly, rendering the area ungovernable is not he worst case scenario being faced. The latter is represented by accelerated sinkhole activity. Although the informal settlement has been geographically stable for some time the decline of mining activity in the larger region has resulted in a rapidly rising water table. Moreover the water now entering the dolomitic deposits is highly acidic - it is the infamous "acid mine water" about which several apocalyptic predictions centre. These two factors (the rising water table and its acidity) promise a marked increase in sinkhole activity - and there is little doubt that Thembalihle residents now face increased risks.
The protests in Thembalihle represent one heritage of poor urban management as Johannesburg Metro Council has never demonstrated the political will to ensure that the area is not used for housing. Resettlements that took place in the past were all accompanied by an immediate resettlement of the land. The failure of adequate urban land management gives rise to the current impasse as grievances cannot be addressed (at least not safely).
Fortunately the disputes in Thembalihle are sure to remain localized, however the same can not be said of the protests at Tembisa.
Tembisa
Tembisa represents a dramatically different situation and has far more wide ranging implications. In Tembisa protests were sparked less by the absence of services than by the poor quality of the services and the cost of those services. Residents cited many grievances but most prominent among their grievances was the rapidly rising cost of electricity. This was blamed, variously, on "overcharging" by the council and the installation of pre-paid meters. The impact of electricity prices and suspicions about Ekhuruleni metro Councils efficacy was most vividly expressed by calls that ESKOM be made the service electricity provider.
This latter demand seems to have been particularly misguided. City councils merely resell electricity bought in bulk from ESKOM. The prices of many services provided by the state (electricity, gas, water, road tolls, license fees) are determined administratively i.e. they are determined by the interaction of supply and demand in a competitive setting but on consideration of other issues. The other considerations include the need to provide free basic services, redress of inequities in service levels, affordability and the need to recoup costs. In other words prices are set by political consideration rather than by markets. Be this as it may the costs of services still have to be recouped from either the fiscus or from other taxes or charges.
In recent years administrative charges have been rising more rapidly than market-related prices. In March this year Business Day gave the example:
"... (in the) last month administered prices rose twice as much as general services and three times as much as the average price of consumer goods".
As ESKOM seeks to fund massive new capital projects increases in the price of electricity has been particularly pronounced. These price increase of approximately 25% each year for four years in a row is not exclusively due to capitalisation costs. They have been prompted by the need to (temporarily at least) reduce electrify consumption and pay very lucrative salaries in the parastatal. Recent reports indicate that the average salary in ESKOM is well in excess of R500 000 per annum. This is more than ten times the value of South Africa' per capita GDP (which, in turn, is higher than average wages). ESKOM is not alone in presenting massive wage bills. In the three years leading to 2010 the total municipal wage bill rose by 54%. Over this period the total number of municipal employees rose by only 5% indicating that average remuneration in municipalities is rising rapidly . All these costs - capitalisation costs, wage bills etc. have to recouped from consumers - hence the pressure of administered prices. These factors ensure that the cost of municipal services like electricity will continue to increase at above the rate of inflation - irrespective of whether the services are provided by the council or another service provider (like ESKOM).
The Tembisa protests indicate that those setting administrative prices have misread the willingness of township residents to bear the ever increasing remuneration and other costs.
Ultimately this is a political misjudgement by the elected representatives and by those appointed to oversee the provision of essential services. If the political representatives have indeed misread the willingness of consumers to absorb ever-increasing costs, as the Tembisa protests imply, then the latest protests may just be the first of many against the price of services. Perhaps we have entered a phase where an increasing proportion of public protests will centre not on the failure to deliver services but on the costs of those services.
What unites the Tembisa and Themblihle protests is the underlying lack of faith residents have in the metropolitan councils. In Thembalihle residents distrust the argument about the impact of dolomite as they witness better services and infrastructure across the road in Lenasia. Moreover they have not recently been affected by sinkholes. Much of the anger in Tembisa has been brought on by the belief that the council is overcharging them for electricity (hence he call for ESKOM to take over). Ultimately the combination of misjudgement by those setting prices and reservations about the integrity of elected representatives suggest that the issue of public order policing will become even more prominent in the years to come.
Sunday, August 29, 2010
SA police, the media and defeating the ends of justice
Media attention has recently been focussed on proposals to exert greater control over it. The proposals include a “media tribunal” and the new Protection of Information bill. In contrast to this concern government agents have defended the proposals as necessary for state security and fairness . Chief spokesman Themba Maseko recently declared that "there is no intention or plan to muzzle the media in any shape or form”, However with attention directed on the explicit attempts to control the “fourth estate” media representative have seemingly been blind to more in insidious moves by the state.
These moves include using existing legislation to achieve the effects envisaged, for example, by the new Protection of Information bill. This bill seeks to criminalise journalists for being in possession of “sensitive” documents that may have been leaked to them. However the recent arrest of Sunday Times journalist Mzilikazi Wa Afrika for, it seems, being in possession of a “fraudulent letter of resignation” suggests that the SA Police Force is already treating the mere possession of sensitive documents as criminal. The initial charges against Afrika were “fraud” and “defeating the ends of justice”.
Although the charge of “defeating the ends of justice” was later dropped in favour of one of “utterance” the fact that this charge was used at all is of concern. “Defeating the ends of justice” is a somewhat vague charge relating to efforts to subvert the judicial process by intimidating witnesses or destroying evidence. If media coverage is anything to go by until recently those charged with the offence were invariably police staff. Examples include Jackie Selebi, the station commissioner who tampered with the drunk driving evidence against Tony Yengeni, former crime intelligence boss Mulangi Mphego and the then head of the Scorpions Gerrie Nel. By contrast it is difficult to find, prior to the appointment of Bheki Cele, reference to the charge being lodged against people outside of the criminal justice system.
In a dramatic departure from this tradition this law is increasingly being used against journalists. In the past two months at least two journalists have been arrested and brought to court for “defeating the ends of justice”. The most recent instance is that of Afrika. In the previous month english journalist Simon Wright faced the charge. The charges against Wright stem from his cloistering of soccer fan Pavlos Joseph in a hotel. During the world cup Josephs entered the English football teams' dressing room to reprimand them for their poor performance. Wright, in his enthusiasm for securing an exclusive interview, booked Josephs into a hotel under a false name. This gave rise to the accusation of him attempting thwart justice from prevailing in Josephs' case. In these instances the veracity the “defeating the ends of justice” charge was justified largely by the vagueness of the allegation. The severity of the charge was aggravated by SAPS intimating, in both instances, that the offences constituted part of a wider conspiracy. The alleged conspiracy, coupled to the vagueness of the charge, invited wild speculation as to what the actual misdeed was.
In court the merits of the “defeating the ends of justice” charges were illustrated by the rapidity with which they were dismissed. The dropping of this charge reflect, on the one hand, the historical difficulty in securing such a conviciton – as was demonstrated in the cases against Selebi , Nel and Mphego. On the other hand the message to journalists remains clear - the police are increasingly prepared to use this charge against them precisely because of the opportunities afforded by its vagueness.
The common theme in both these cases is that the journalists had recently embarrassed the SAPS. Afrika had just accused police head Beki Cele of signing a R500-million lease without observing the required procedures. Wright had pointed out the ease with which a fan could wander into a key area secured by the SAPS. Could it be that the arrests were to punish this behaviour rather than any overt criminal act?
In a short period “defeating the ends of justice” has come to be associated less with tampering with evidence or bribing witnesses and more with journalists publishing the true face of the police.
The extent to which the police hereby undermine the fourth estate is the extent to which they may not need the full measure of the Protection of Information bill. What their action does show is that, assurance that "there is no intention or plan to muzzle the media in any shape or form” aside, the state seeks to cow the media irrespective of whether or not the Protection of Information bill is passed in its current form. Those who view the proponents of the protection of information bill as naïve but well intentioned people operating a bit out of their depth should wake up. In this one regard the writing (on the wall) is getting clearer.
WOL (wake-on-lan) over the Internet
You learn about internet sites that allow you to use wake-on-lan (WOL) over the Internet and have a wake-on-lan (WOL) enabled PC. You now reckon you can wake your home PC from a remote internet location. From that location you can then use SSH, telnet, or tightvnc to access your PC subject, of course, to the limitations of internet speed. When you try this you will soon discover that it does not work.
Unfortunately WOL requires that at least part of a LAN must be awake. If you put your PC off it, to all intents and purposes, ceases to exist on the web. Under these circumstances the WOL-over-internet providers like www.depicus.com and www.wakeonlan.me will work for up to 30 minutes after you powered off. This is obviously not what is needed.
However your ADSL router is part of your home LAN and it remains on all the time. So to use WOL over the internet you need to set things up so that your ADSL router can send a "magic packet" to the PC. This magic packet will then wake the PC. To do this the following must happen.
1) Your motherboard and the relevant network card needs to support WOL.
2) You must be able to "http" into your ADSL router from a remote location.
3) You must be able to administer remotely (in order to send the WOL packet)
4) You must be able to tell the router what the IP and MAC addresses of the network card are.
4) Your router must be able to send a WOL "magic packet" to that address.
In detail:
1) is a no-brainer. Either your PC does WOL or it doesn't. If it does use the BIOS setting to enable WOL. This usullay requires you pressing some key (usually DEL) before the computer boots.
2) The usual way of changing router settings is via your browser. For example, my Linksys WAG2000 requires that I go to http://192.168.1.1 and enter the username and password written on the underside of the router. However this address only works if the PC is linked directly to that router.
To be able to link to the router remotely (i.e. form elsewhere on the internet) you need to subscribe to a service like "DynDNS.com" or "no-ip.com". These two services are free and provide you with a domain name that you can use to http into your router from anywhere. After obtaining your address make the appropriate changes on your router and reboot it.
There are two changes that need to be made.
1) The router must be told that you are using a DNS system like DynDNS. On the Linksys this is under Setup - DDNS. This links your router to a fixed ip address like mine (maodonovxxx.yyy.zz).
2) The router must be enabled for remote adminstration. On the Linksys this is under Administration - Management - Remote Gateway Access. All you have to do is enable the facility and record the port number.
Once you have saved the settings and rebooted your router you can access it via secure http. In my instance is go to
https://maodonovxxx.yyy.zz:8080
NOTE there is a "s" in the address above.
Remember to change your router password to something different from the default. If I had not done that you could http into my router and change my settings.
3) To access the WOL network card from the router you have to be able to identify it by IP and Hardware address (MAC). On a linus (Debian) system you would (as superuser) run ifconfig. My system says for network card eth0:
eth0
Link encap:Ethernet HWaddr 00:21:85:61:da:9e
inet addr:192.168.1.4 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::221:85ff:fe61:da9e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11415 errors:0 dropped:0 overruns:0 frame:0
TX packets:9971 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:9922062 (9.4 MiB) TX bytes:1663804 (1.5 MiB)
Interrupt:250 Base address:0x4000
(On Windows run ipconfig.)
Note your inet addr (the IP), Bcast (Broadcast), HWadd (i.e. the MAC) and Mask.
If you are using a dynamic addressing system (DHCP) the IP address will change regularly and you will not know what IP address to enter. So use a static address for the hardware card. On linux (Debian) systems you will do this by editing /etc/network/interfaces. On Windows you will presumably be able to change the settings by right clicking on the network card.
If you do not know what setting to use try those published after running ifconfig (*nix) or ipconfig (Windows). Remember to restart your network (Debian uses "/etc/init.d/networking restart")
4) Theoretically you could now http into your router from the internet and issue a wake-on-lan command to the IP and MAC address you just set. However chances are that your router has not provided any WOL facility. This is certainly the case for Cisco and Netgear. Fortunately the OpenSource community has provided the required software for many routers, This software is model specific so do not use software for another brand or model router.
Relevant sources are:
Netgear http://dgteam.ilbello.com/
Linksys http://sourceforge.net/projects/openwag200/
Other routers http://openwrt.org or http://google.com
The process of installing the software is one of downloading a zipped/ compressed file, uncompressing it and then updating the "firmware" on the router. Your router will already have update facility. Although flashing your router is simple read the instructions (and warnings) provided. Although the upgrade preserves settings it does make sense to back them up to your hard drive first.
Once flashed and rebooted you can http into the router. You will now have a new menu item under which various new services are listed. On OpenWAG (used for the LinkSys WAG200) it is - "MySetup".
Negotiate your way to the WOL facility. On OPenWAG it is under:
MySetup - other - Wake on LAN Interface.
Here you enter the MAC address (from ifconfig/ipconfig) and the IP address. This is the IP address of your LAN and not the IP provided by no-ip or DynDNS. In my case it is 192.168.1.4.
Press "save" and a magic packet is sent to the PC powering it on.
..... thats it
Unfortunately WOL requires that at least part of a LAN must be awake. If you put your PC off it, to all intents and purposes, ceases to exist on the web. Under these circumstances the WOL-over-internet providers like www.depicus.com and www.wakeonlan.me will work for up to 30 minutes after you powered off. This is obviously not what is needed.
However your ADSL router is part of your home LAN and it remains on all the time. So to use WOL over the internet you need to set things up so that your ADSL router can send a "magic packet" to the PC. This magic packet will then wake the PC. To do this the following must happen.
1) Your motherboard and the relevant network card needs to support WOL.
2) You must be able to "http" into your ADSL router from a remote location.
3) You must be able to administer remotely (in order to send the WOL packet)
4) You must be able to tell the router what the IP and MAC addresses of the network card are.
4) Your router must be able to send a WOL "magic packet" to that address.
In detail:
1) is a no-brainer. Either your PC does WOL or it doesn't. If it does use the BIOS setting to enable WOL. This usullay requires you pressing some key (usually DEL) before the computer boots.
2) The usual way of changing router settings is via your browser. For example, my Linksys WAG2000 requires that I go to http://192.168.1.1 and enter the username and password written on the underside of the router. However this address only works if the PC is linked directly to that router.
To be able to link to the router remotely (i.e. form elsewhere on the internet) you need to subscribe to a service like "DynDNS.com" or "no-ip.com". These two services are free and provide you with a domain name that you can use to http into your router from anywhere. After obtaining your address make the appropriate changes on your router and reboot it.
There are two changes that need to be made.
1) The router must be told that you are using a DNS system like DynDNS. On the Linksys this is under Setup - DDNS. This links your router to a fixed ip address like mine (maodonovxxx.yyy.zz).
2) The router must be enabled for remote adminstration. On the Linksys this is under Administration - Management - Remote Gateway Access. All you have to do is enable the facility and record the port number.
Once you have saved the settings and rebooted your router you can access it via secure http. In my instance is go to
https://maodonovxxx.yyy.zz:8080
NOTE there is a "s" in the address above.
Remember to change your router password to something different from the default. If I had not done that you could http into my router and change my settings.
3) To access the WOL network card from the router you have to be able to identify it by IP and Hardware address (MAC). On a linus (Debian) system you would (as superuser) run ifconfig. My system says for network card eth0:
eth0
Link encap:Ethernet HWaddr 00:21:85:61:da:9e
inet addr:192.168.1.4 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::221:85ff:fe61:da9e/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:11415 errors:0 dropped:0 overruns:0 frame:0
TX packets:9971 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:9922062 (9.4 MiB) TX bytes:1663804 (1.5 MiB)
Interrupt:250 Base address:0x4000
(On Windows run ipconfig.)
Note your inet addr (the IP), Bcast (Broadcast), HWadd (i.e. the MAC) and Mask.
If you are using a dynamic addressing system (DHCP) the IP address will change regularly and you will not know what IP address to enter. So use a static address for the hardware card. On linux (Debian) systems you will do this by editing /etc/network/interfaces. On Windows you will presumably be able to change the settings by right clicking on the network card.
If you do not know what setting to use try those published after running ifconfig (*nix) or ipconfig (Windows). Remember to restart your network (Debian uses "/etc/init.d/networking restart")
4) Theoretically you could now http into your router from the internet and issue a wake-on-lan command to the IP and MAC address you just set. However chances are that your router has not provided any WOL facility. This is certainly the case for Cisco and Netgear. Fortunately the OpenSource community has provided the required software for many routers, This software is model specific so do not use software for another brand or model router.
Relevant sources are:
Netgear http://dgteam.ilbello.com/
Linksys http://sourceforge.net/projects/openwag200/
Other routers http://openwrt.org or http://google.com
The process of installing the software is one of downloading a zipped/ compressed file, uncompressing it and then updating the "firmware" on the router. Your router will already have update facility. Although flashing your router is simple read the instructions (and warnings) provided. Although the upgrade preserves settings it does make sense to back them up to your hard drive first.
Once flashed and rebooted you can http into the router. You will now have a new menu item under which various new services are listed. On OpenWAG (used for the LinkSys WAG200) it is - "MySetup".
Negotiate your way to the WOL facility. On OPenWAG it is under:
MySetup - other - Wake on LAN Interface.
Here you enter the MAC address (from ifconfig/ipconfig) and the IP address. This is the IP address of your LAN and not the IP provided by no-ip or DynDNS. In my case it is 192.168.1.4.
Press "save" and a magic packet is sent to the PC powering it on.
..... thats it
Sunday, June 20, 2010
Weighted mean: tapply
To obtain weighted means for categories of a dataframe use has to be made of
a) the tapply function and
b) the weighted.mean function.
The information required for the tapply component is:
"lll" - a list of the rows to which the function must be applied
"rrr" - the categories over which the function must be aggregated
"function(i, x, w,)...." - the function to apply. In this instance the 'weighted mean'.
The info required for the weighted.mean component are 'x' (the values to be weighted) and 'w' (the weights).
The general form is:
tapply(seq(along=lll), rrr, function(i, x, w) weighted.mean(x[i], w[i]),x=lll, w=ttt)
An example of weighting incomes ("incval") by frequency counts ("worker_wgt") for each "tbvc" category is:
tapply(seq(along=incval),tbvc, function(i, x, w) {weighted.mean(incval[i], worker_wgt[i],na.rm=T)})
Obviously this is done after attaching the relevant dataframe.
a) the tapply function and
b) the weighted.mean function.
The information required for the tapply component is:
"lll" - a list of the rows to which the function must be applied
"rrr" - the categories over which the function must be aggregated
"function(i, x, w,)...." - the function to apply. In this instance the 'weighted mean'.
The info required for the weighted.mean component are 'x' (the values to be weighted) and 'w' (the weights).
The general form is:
tapply(seq(along=lll), rrr, function(i, x, w) weighted.mean(x[i], w[i]),x=lll, w=ttt)
An example of weighting incomes ("incval") by frequency counts ("worker_wgt") for each "tbvc" category is:
tapply(seq(along=incval),tbvc, function(i, x, w) {weighted.mean(incval[i], worker_wgt[i],na.rm=T)})
Obviously this is done after attaching the relevant dataframe.
Thursday, October 29, 2009
Extract variables from data frame by name
I had to combine a bunch of .csv files into a data frame. As the various files had no column names a side effect of the import were duplicate column names. A similar result could arise when using "cbind". As I was only interested in the eighth variable (called "V8" by default) I needed a way to extract all (and only) the V8's into a new data frame. Unfortunately the "V8"s were not at regular intervals.
names(april2008) yields:
[1] "station" "V7" "V8" "V4" "V5" "V6" "V7"
[8] "V8" "V5" "V6" "V7" "V8" "V6" "V7"
[15] "V8
I wanted to extract columns 3, 8, 12, 15.
This was achieved by using the "which" command. The initial data frame was called "april2008".
I used:
april2008[,which(names(april2008)=="V8"] -> newapril08
names(newapril08) yields:
[1] "V8" "V8.1" "V8.2" "V8.3"
names(april2008) yields:
[1] "station" "V7" "V8" "V4" "V5" "V6" "V7"
[8] "V8" "V5" "V6" "V7" "V8" "V6" "V7"
[15] "V8
I wanted to extract columns 3, 8, 12, 15.
This was achieved by using the "which" command. The initial data frame was called "april2008".
I used:
april2008[,which(names(april2008)=="V8"] -> newapril08
names(newapril08) yields:
[1] "V8" "V8.1" "V8.2" "V8.3"
Monday, October 12, 2009
sorting a dataframe
To order selected variables in a dataframe you need to use the "order" and not the "sort" function. The easiest way to do this is to use the numerical (square bracket) notation system.
Specify the dataframe (lets call it "stations") and in the place where you would usually specify which rows to use place the command....
order(variable to order by).
So ..
station[order(station$V4),2:4]
will produce a data frame composed of the 2nd 3rd and 4th variables of the station dataset. However it will now be ordered by the variable called "V4".
Specify the dataframe (lets call it "stations") and in the place where you would usually specify which rows to use place the command....
order(variable to order by).
So ..
station[order(station$V4),2:4]
will produce a data frame composed of the 2nd 3rd and 4th variables of the station dataset. However it will now be ordered by the variable called "V4".
Monday, September 28, 2009
Import multiple files into R
I needed to import 144 files in comma separated variable (.csv) format into R. Each individual imported file was to be a dataframe. Obviously, given the nuber of files, automating the process was highly desirable. It took quite a bit of searching before I found a simple solution. The biggest "problem" was finding a way of generating the names of the new R objects. The solution:
Create an R object (called "filelist") giving the names of all the .csv files in the relevant directory.
list.files("/name/the/directory/",pattern='csv$', full.names=T) -> filelist
The "pattern" command lists all files ending in ".csv".
Now its possible to import each of these files using a simple "for" loop. However the point is to reference each cvs file not by its name but by its location in the above "filelist" object. This variable will also be used to generate the name of each of the 144 new dataframes. These will be called named between "thisit1" and "thisit144".
for (var in 1:144) assign(paste("thisit",var,sep=""),read.csv(filelist[var]))
Create an R object (called "filelist") giving the names of all the .csv files in the relevant directory.
list.files("/name/the/directory/",pattern='csv$', full.names=T) -> filelist
The "pattern" command lists all files ending in ".csv".
Now its possible to import each of these files using a simple "for" loop. However the point is to reference each cvs file not by its name but by its location in the above "filelist" object. This variable will also be used to generate the name of each of the 144 new dataframes. These will be called named between "thisit1" and "thisit144".
for (var in 1:144) assign(paste("thisit",var,sep=""),read.csv(filelist[var]))
Subscribe to:
Posts (Atom)
