Converting Hex To Decimal In Awk Or Sed


Answer :

Here's a variation on Jonathan's answer:



awk $([[ $(awk --version) = GNU* ]] && echo --non-decimal-data) -F, '
    BEGIN {OFS = FS}
    {
        $6 = sprintf("%d", "0x" substr($4, 11, 4))
        $5 = sprintf("%d", "0x" substr($4,  7, 4))
        $4 = substr($4,  1, 6)
        print
    }'


I included a rather contorted way of adding the --non-decimal-data option if it's needed.



Edit



Just for the heck of it, here's the pure-Bash equivalent:



saveIFS=$IFS
IFS=,
while read -r -a line
do
    printf '%s,%s,%d,%d\n' "${line[*]:0:3}" "${line[3]:0:6}" "0x${line[3]:6:4}" "0x${line[3]:10:4}"
done
IFS=$saveIFS


The "${line[*]:0:3}" (quoted *) works similarly to AWK's OFS in that it causes Bash's IFS (here a comma) to be inserted between array elements on output. We can take further advantage of that feature by inserting array elements as follows which more closely parallels my AWK version above.



saveIFS=$IFS
IFS=,
while read -r -a line
do
    line[6]=$(printf '%d' "0x${line[3]:10:4}")
    line[5]=$(printf '%d' "0x${line[3]:6:4}")
    line[4]=$(printf '%s' "${line[3]:0:6}")
    printf '%s\n' "${line[*]}"
done
IFS=$saveIFS


Unfortunately, Bash doesn't allow printf -v (which is similar to sprintf()) to make assignments to array elements, so printf -v "line[6]" ... doesn't work.



Edit: As of Bash 4.1, printf -v can now make assignments to array elements. Example:



printf -v 'line[6]' '%d' "0x${line[3]:10:4}"


The quotes around the array reference are needed to prevent possible filename matching. If a file named "line6" existed in the current directory and the reference wasn't quoted, then a variable named line6 would be created (or updated) containing the printf output. Nothing else about the file, such as its contents, would come into play. Only the name - and only tangentially.



This seems to work:



awk -F, '{ p1 =       substr($4,  1, 6);
           p2 = ("0x" substr($4,  7, 4)) + 0;
           p3 = ("0x" substr($4, 11, 4)) + 0;
           printf "%s,%s,%s,%s,%d,%d\n", $1, $2, $3, p1, p2, p3;
         }'


For your sample input data, it produces:



123711184642,02,3583090366663629,639f02,292,14292
123715942138,01,3538710295145500,639f02,45014,50755
123711616258,02,3548370476972758,639f02,72,22322


The string concatenation of '0x' plus the 4-digit hex followed by adding 0 forces awk to treat the numbers as hexadecimals.



You can simplify this to:



awk -F, '{ p1 =      substr($4,  1, 6);
           p2 = "0x" substr($4,  7, 4);
           p3 = "0x" substr($4, 11, 4);
           printf "%s,%s,%s,%s,%d,%d\n", $1, $2, $3, p1, p2, p3;
         }'


The strings prefixed with 0x are forced to integer when presented to printf() and the %d format.





The code above works beautifully with the native awk on MacOS X 10.6.5 (version 20070501); sadly, it does not work with GNU gawk 3.1.7. That, it seems, is permitted behaviour according to POSIX (see the comments below). However, gawk has a non-standard function strtonum that can be used to bludgeon it into performing correctly - pity that bludgeoning is necessary.



gawk -F, '{ p1 =      substr($4,  1, 6);
            p2 = "0x" substr($4,  7, 4);
            p3 = "0x" substr($4, 11, 4);
            printf "%s,%s,%s,%s,%d,%d\n", $1, $2, $3, p1, strtonum(p2), strtonum(p3);
          }'


By AWK



This answer concentrates on showing how to do the conversion by awk portably.



Using --non-decimal-data for gawk is not recommended according to GNU Awk User's Guide. And using strtonum() is not portable.



In the following examples the first word of each record is converted.



By user-defined function



The most portable way of doing conversion is by a user-defined awk function [reference]:



function parsehex(V,OUT)
{
    if(V ~ /^0x/)  V=substr(V,3);

    for(N=1; N<=length(V); N++)
        OUT=(OUT*16) + H[substr(V, N, 1)]

    return(OUT)
}

BEGIN { for(N=0; N<16; N++)
        {  H[sprintf("%x",N)]=N; H[sprintf("%X",N)]=N } }

{ print parsehex($1) }


By calling shell's printf



You could use this



awk '{cmd="printf %d 0x" $1; cmd | getline decimal; close(cmd); print decimal}'


but it is relatively slow. The following one is faster, if you have many newline-separated hexadecimal numbers to convert:



awk 'BEGIN{cmd="printf \"%d\n\""}{cmd=cmd " 0x" $1}END{while ((cmd | getline dec) > 0) { print dec }; close(cmd)}'


There might be a problem if very many arguments are added for the single printf command.



In Linux



In my experience the following works in Linux:



awk -Wposix '{printf("%d\n","0x" $1)}'


I tested it by gawk, mawk and original-awk in Ubuntu Linux 14.04. By original-awk the command displays a warning message, but you can hide it by redirection directive 2>/dev/null in shell. If you don't want to do that, you can strip the -Wposix in case of original-awk like this:



awk $(awk -Wversion >/dev/null 2>&1 && printf -- "-Wposix") '{printf("%d\n","0x" $1)}'


(In Bash 4 you could replace >/dev/null 2>&1 by &>/dev/null)



Note: The -Wposix trick probably doesn't work with nawk which is used in OS X and some BSD OS variants, though.



Comments

Post a Comment

Popular posts from this blog

Converting A String To Int In Groovy

Answer : Use the toInteger() method to convert a String to an Integer , e.g. int value = "99".toInteger() An alternative, which avoids using a deprecated method (see below) is int value = "66" as Integer If you need to check whether the String can be converted before performing the conversion, use String number = "66" if (number.isInteger()) { int value = number as Integer } Deprecation Update In recent versions of Groovy one of the toInteger() methods has been deprecated. The following is taken from org.codehaus.groovy.runtime.StringGroovyMethods in Groovy 2.4.4 /** * Parse a CharSequence into an Integer * * @param self a CharSequence * @return an Integer * @since 1.8.2 */ public static Integer toInteger(CharSequence self) { return Integer.valueOf(self.toString().trim()); } /** * @deprecated Use the CharSequence version * @see #toInteger(CharSequence) */ @Deprecated public static Integer toInteger(String self) { return toInteg
Read more

"Cannot Create Cache Directory /home//.composer/cache/repo/https---packagist.org/, Or Directory Is Not Writable. Proceeding Without Cache"

Answer : if anyone pass through here, this is shorter solution: sudo chown -R $USER $HOME/.composer it seems to me the group information is missing in your command sudo chown -R <user> /home/<user>/.composer/cache/repo/https---packagist.org Shoud be sudo chown -R <user>:<group> /home/<user>/.composer/cache/repo/https---packagist.org But to avoid other permission issues, I would rather advise: sudo chown -R <user>:<group> /home/<user>/.composer/cache (you'll need access to other folders in there) and sudo chown <user>:<group> /home/<user>/.composer To make sure your user has permissions enough on the global composer folder. Mind the missing recursion so the user don't own keys created by root. If you need to find out the group: groups <user>
Read more

Android How Can I Convert A String To A Editable

Answer : As you probably found out, Editable is an interface so you cannot call new Editable() . However an Editable is nothing more than a String with Spannables, so use SpannableStringBuilder: Editable editable = new SpannableStringBuilder("Pass a string here"); If you only want to change the Spannables, never the text itself, you can use a basic SpannableString. Use Editable.Factory.getInstance().newEditable(str) From the android documentation: Returns a new SpannedStringBuilder from the specified CharSequence. You can override this to provide a different kind of Spanned.
Read more