Back to developer Blog

Converting to plain ASCII

  • February 20, 2017
  • Magnus Carlsson

This is an update of an older blog post and is now taking advantage of the Python canlib package v1.7.

This is the second post in a 3-part series about using the Converter Library (kvlclib) in CANlib SDK:

  1. Writer formats and properties (1 of 3)
  2. Converting to plain ASCII (2 of 3)
  3. Special converter conditions (3 of 3)

The first part of this blog series looked into converter formats and properties, we will now continue and convert events from a given log file.

Let us assume that we have a log file, mylog.kme50, that we would like to convert to Plain text (.txt) format. The log file was created either by using the Kvaser Memorator Config Tool, or by using the kvmlib as described in an earlier blog post.1

Starting with the last code example we wrote in the previous blog article in this series, we now add setting up the name and format of our input file at the end.

# set up formats
out_fmt = kvlclib.WriterFormat(kvlclib.FileFormat.PLAIN_ASC)
in_fmt = kvlclib.ReaderFormat(kvlclib.FileFormat.KME50)

# set resulting output file name taking advantage of the extension
# defined in the format.
out_file = "myresult." + out_fmt.extension
print("Output filename is '%s'" % out_file)

# create converter
cnv = kvlclib.Converter(out_file, out_fmt)

# Set input file and format
cnv.setInputFile(INPUT_FILE, kvlclib.FileFormat.KME50)

Let us also add a loop that converts the events one by one until we reach the end of our input file. The last thing we do is to call Converter.flush() in order to flush output to disk.

# Convert events from input file one by one until EOF is reached
while True:
    try:
        cnv.convertEvent()
    except kvlclib.KvlcEndOfFile:
        break

# Force a flush so that the resulting file is written to disk.
cnv.flush()

Our first try at writing a kme50 converter program now looks as follows.

# 04_first_try_converting.py
from canlib import kvlclib


INPUT_FILE = 'mylog.kme50'

# set up formats
out_fmt = kvlclib.WriterFormat(kvlclib.FileFormat.PLAIN_ASC)
in_fmt = kvlclib.ReaderFormat(kvlclib.FileFormat.KME50)

# set resulting output file name taking advantage of the extension
# defined in the format.
out_file = "myresult." + out_fmt.extension
print("Output filename is '%s'" % out_file)

# create converter
cnv = kvlclib.Converter(out_file, out_fmt)

# Set input file and format
cnv.setInputFile(INPUT_FILE, kvlclib.FileFormat.KME50)

# Convert events from input file one by one until EOF is reached
while True:
    try:
        cnv.convertEvent()
    except kvlclib.KvlcEndOfFile:
        break

# Force a flush so that the resulting file is written to disk.
cnv.flush()

Running the above code, assuming everything works as expected, the file myresult.txt is created. But running the same code again results in an error:

canlib.kvlclib.exceptions.KvlcGeneralError: Output file already exists (-6)

Each converter has a number of properties that effect their inner workings. One of these properties is OVERWRITE which defaults to ‘0’, and that is the reason the code failed when running the second time – the resulting file already existed and the converter was not allowed to overwrite it.

As we saw in the previous blog post, we can ask if a property is supported by a specific format. So let us create a function that checks if a given property is supported. If the property is supported – and a value was supplied to the function – the function sets the property to the given value. We also take the opportunity to print the default value of the property to the console.

def trySetProperty(cnv, property, value=None):
    # Check if the format supports the given property
    if cnv.format.isPropertySupported(property):
        # If a value is specified, set the property to this value
        if value is not None:
            cnv.setProperty(property, value)

        # Get the property's default value
        default = cnv.format.getPropertyDefault(property)
        print('  %s is supported (Default: %s)' %
              (property, default))

        # Get the property's current value
        value = cnv.getProperty(property)
        print('    Current value: %s' % value)
    else:
        print('  %s is not supported' % property)

Now let us use the trySetProperty() function and set the property OVERWRITE to ‘1’, we also limit the output to only the first channel by setting the CHANNEL_MASK property. Another useful property for the Plain text format is WRITE_HEADER which adds a nice header to the output file, so let us set that property as well. The final property we set is LIMIT_DATA_BYTES which controls how many data bytes to print. In the ASCII format this directly affects how wide our data table will be, and since we do not have any CAN FD2 messages in our input log, we set this property to 8 (bytes).

# allow output file to overwrite existing files
trySetProperty(cnv, kvlclib.Property.OVERWRITE, 1)

# we are only interested in the first channel
cnv.setProperty(kvlclib.Property.CHANNEL_MASK, 1)

# add nice header to the output file
trySetProperty(cnv, kvlclib.Property.WRITE_HEADER, 1)

# we are converting CAN traffic with max 8 bytes, so we can minimize
# the width of the data output to 8 bytes
trySetProperty(cnv, kvlclib.Property.LIMIT_DATA_BYTES, 8)

Collecting all our improvements so far, we end up with the following program.

# 05_convert_kme.py
from canlib import kvlclib

def trySetProperty(cnv, property, value=None):
    # Check if the format supports the given property
    if cnv.format.isPropertySupported(property):
        # If a value is specified, set the property to this value
        if value is not None:
            cnv.setProperty(property, value)

        # Get the property's default value
        default = cnv.format.getPropertyDefault(property)
        print('  %s is supported (Default: %s)' %
              (property, default))

        # Get the property's current value
        value = cnv.getProperty(property)
        print('    Current value: %s' % value)
    else:
        print('  %s is not supported' % property)

# set up formats
out_fmt = kvlclib.WriterFormat(kvlclib.FileFormat.PLAIN_ASC)
in_fmt = kvlclib.ReaderFormat(kvlclib.FileFormat.KME50)

# set resulting output file name taking advantage of the extension
# defined in the format.
out_file = "myresult." + out_fmt.extension
print("Output filename is '%s'" % out_file)

# create converter
cnv = kvlclib.Converter(out_file, out_fmt)

# Set input file and format
cnv.setInputFile(INPUT_FILE, kvlclib.FileFormat.KME50)

# allow output file to overwrite existing files
trySetProperty(cnv, kvlclib.Property.OVERWRITE, 1)

# we are only interested in the first channel
cnv.setProperty(kvlclib.Property.CHANNEL_MASK, 1)

# add nice header to the output file
trySetProperty(cnv, kvlclib.Property.WRITE_HEADER, 1)

# we are converting CAN traffic with max 8 bytes, so we can minimize
# the width of the data output to 8 bytes
trySetProperty(cnv, kvlclib.Property.LIMIT_DATA_BYTES, 8)

# Convert events from input file one by one until EOF is reached
while True:
    try:
        cnv.convertEvent()
    except kvlclib.KvlcEndOfFile:
        break

# Force a flush so that the resulting file is written to disk.
cnv.flush()

Running the full program results in the following output.

                              Kvaser Memorator Log
                              ====================

Converted from Memorator Binary format at: 2018-08-14 07:47:10

                                                        
Settings:
   Format of data field: DEC
   Format of id field:   DEC
   Timestamp Offset:     0.000000 s
   CAN channel:          1

       Time Chan   Identifier Flags     DLC  Data                            Counter
====================================================================================
   0.242162  Trigger (type=0x1, active=0x00, pre-trigger=0, post-trigger=-1)
  32.003407  1         501    Rx         8   41  23   0   0   0   0   0   0        2
  32.006923  1         501    Rx         8  234   0   0   0   0   0   0   0        4
  32.007795  1         504    Rx         8   51 179  79  67   0   0   0   0        6
  32.008373  1         402    Rx         8   51  51 134  66   0   0   0   0        8
  32.009127  1         402    Rx         8   51  51 243  64   0   0   0   0       10
  32.009925  1         504    Rx         8    0   0  56  66   0   0   0   0       12
  32.010670  1         503    Rx         8    0   0  52  67   0   0   0   0       14
  32.011413  1         504    Rx         8  154  25  90  67   0   0   0   0       16
  32.012167  1         402    Rx         8  205 204 140  64   0   0   0   0       18
  32.015623  1         504    Rx         8  205 204  68  65   0   0   0   0       20

This ends the second blog article in which we created a simple program that used the Kvaser Converter library, kvlclib, to convert a kme50 log file into a plain text file. In the last article of this series, we will look at some special conditions that can arise when converting a log file.

If you have any questions, comments or suggestion for future blog articles, you can contact us directly via mail at [email protected].


Footnotes

1 Reading from a Kvaser Memorator and saving the logged data using kvmlib was done in www.kvaser.com/developer-blog/digging-deeper-into-kvmlib

2 CAN FD allows data fields up to 64 bytes per frame. See the blog article www.kvaser.com/developer-blog/can-fd for more information about CAN FD.


This article has been updated. To view the original, click on the box below.

Original Article

This blog uses now deprecated functions, see blog “Improved API in Python canlib v1.5” for more information.

This is the second post in a 3-part series about using the Converter Library (kvlclib) in CANlib SDK:

  1. Writer formats and properties (1 of 3)
  2. Converting to plain ASCII (2 of 3)
  3. Special converter conditions (3 of 3)

The first part of this blog series looked into converter formats and properties, we will now continue and convert events from a given log file.

Let us assume that we have a log file, mylog.kme50, that we would like to convert to Plain text (.txt) format. The log file was created either by using the Kvaser Memorator Config Tool, or by using the kvmlib as described in an earlier blog post.[1]

Starting with the latest code sample we wrote in the first part of this blog series, we now add some code for setting up the format and filename of our input file at the end.

import canlib.kvlclib as kvlc

# set output format
fmt = kvlc.WriterFormat(kvlc.FILE_FORMAT_PLAIN_ASC)
# the name of the formatter is fetched using kvlcGetWriterName() internally
print("Output format is '%s'" % fmt.name)

# set resulting output filename taking advantage of the extension defined in
# the format. (Uses kvlcGetWriterExtension() under the hood.)
outfile = "myresult." + fmt.extension
print("Output filename is '%s'" % outfile)

# create converter
kc = kvlc.Kvlclib(outfile, fmt)

# Set input filename and format
inputfile = "mylog.kme50"
print("Input filename is '%s'" % inputfile)
kc.setInputFile(inputfile, file_format=kvlc.FILE_FORMAT_KME50)

Let us now add a loop that converts the events one by one until we reach the end of our input file.

# Convert events from input file one by one until EOF is reached
while True:
    try:
        kc.convertEvent()
    except kvlc.KvlcEndOfFile:
        break

The last thing we need to do is to call kvlcDeleteConverter() in order to flush output to disk and free memory.

# Delete converter, flush result to disk and free memory
kc.deleteConverter()

Running the above code once, the file myresult.txt is created. But running the same code again results in an error:

  File "c:\dev\python\pycanlib\src\canlib\kvlclib.py", line 459, in convertEvent
    self.dll.kvlcConvertEvent(self.handle)
  File "c:\dev\python\pycanlib\src\canlib\kvlclib.py", line 408, in _kvlc_err_check
    raise KvlcError(self, result)
canlib.kvlclib.KvlcError: [KvlcError] convertEvent: Output file already exists (-6)

Each converter has a number of properties that effect their inner workings. One of these properties is KVLC_PROPERTY_OVERWRITE which defaults to ’0’, and that is the reason the code failed when running the second time – the resulting file already existed and the converter was not allowed to overwrite the existing file.

As we saw in the previous blog post, we can ask if a property is supported by a specific format. So let us create a function that checks if a given property is supported. If the property is supported – and a value was supplied to the function – the function sets the property to the given value. We also take the opportunity to print the default value of the property to the console.

def trySetProperty(converter, property, value=None):
    # Check if the format supports the given property
    if converter.format.isPropertySupported(property):

        # If a value was specified, set the property to this value
        if value is not None:
            converter.setProperty(property, value)

        # get the property's default value
        default = converter.getPropertyDefault(property)
        print(" PROPERTY_%s is supported (Default: %s)" %
              (property['name'], default))

        # get the property's current value
        value = converter.getProperty(property)
        print("	Current value: %s" % value)
    else:
        print(" PROPERTY %s is not supported" %
              (property['name']))

Listing 4: Defining a function that tries to set a given property.

Now let us use the trySetProperty() function defined in Listing 4 and set the KVLC_PROPERTY_OVERWRITE to ‘1’. A second useful property for the Plain text format is KVLC_PROPERTY_WRITE_HEADER which adds a nice header to the output file so let us set that property as well. The final property we set is KVLC_PROPERTY_LIMIT_DATA_BYTES which controls how many data bytes to print. In the ASCII format this directly affects how wide our data table will be, and since we do not have any CAN FD[2] messages in our input log, we set this property to 8 (bytes).

# allow output file to overwrite existing files
trySetProperty(kc, kvlc.PROPERTY_OVERWRITE, 1)

# add nice header to the output file
trySetProperty(kc, kvlc.PROPERTY_WRITE_HEADER, 1)

# we are converting CAN traffic with max 8 bytes, so we can minimize the width
# of the data output to 8 bytes
trySetProperty(kc, kvlc.PROPERTY_LIMIT_DATA_BYTES, 8)

Listing 5: Setting some convenient properties when using the plain text writer.

Collecting all our improvements so far, we end up with the program shown in Listing 6.

import canlib.kvlclib as kvlc


def trySetProperty(converter, property, value=None):
    # Check if the format supports the given property
    if converter.format.isPropertySupported(property):

        # If a value was specified, set the property to this value
        if value is not None:
            converter.setProperty(property, value)

        # get the property's default value
        default = converter.getPropertyDefault(property)
        print(" PROPERTY_%s is supported (Default: %s)" %
              (property['name'], default))

        # get the property's current value
        value = converter.getProperty(property)
        print("	Current value: %s" % value)
    else:
        print(" PROPERTY %s is not supported" %
              (property['name']))


# set output format
fmt = kvlc.WriterFormat(kvlc.FILE_FORMAT_PLAIN_ASC)
# the name of the formatter is fetched using kvlcGetWriterName() internally
print("Output format is '%s'" % fmt.name)

# set resulting output filename taking advantage of the extension defined in
# the format. (Uses kvlcGetWriterExtension() under the hood.)
outfile = "myresult." + fmt.extension
print("Output filename is '%s'" % outfile)

# create converter
kc = kvlc.Kvlclib(outfile, fmt)

# Set input filename and format
inputfile = "mylog.kme50"
print("Input filename is '%s'" % inputfile)
kc.setInputFile(inputfile, file_format=kvlc.FILE_FORMAT_KME50)

# allow output file to overwrite existing files
trySetProperty(kc, kvlc.PROPERTY_OVERWRITE, 1)

# add nice header to the output file
trySetProperty(kc, kvlc.PROPERTY_WRITE_HEADER, 1)

# we are converting CAN traffic with max 8 bytes, so we can minimize the width
# of the data output to 8 bytes
trySetProperty(kc, kvlc.PROPERTY_LIMIT_DATA_BYTES, 8)

# Convert events from input file one by one until EOF is reached
while True:
    try:
        kc.convertEvent()
    except kvlc.KvlcEndOfFile:
        break

# Delete converter, flush result to disk and free memory
kc.deleteConverter()

Listing 6: Simple program to convert a kme50 log file to plain text output.

Our second attempt as shown in Listing 6 can be run multiple times, and the top part of the generated file myresult.txt is shown in Listing 7.

                               Kvaser Memorator Log
                               ====================

 Converted from Memorator Binary format at: 1/11/2017 15:54:02


Settings:
    Format of data field: DEC
    Format of id field:   DEC
    Timestamp Offset:     0.000000000 s
    CAN channel:          1 2

        Time Chan  Identifier Flags        DLC   Data                                     Counter
=============================================================================================
 0.247909525 Trigger (type=0x1, active=0x00, pre-trigger=0, post-trigger=-1)
10.648166375 1          55    Rx            1   2                                           1
10.648166375 2          55    Rx            1   2                                           2
12.464742375 1         340    Rx            8   0   0   0   0   0   0   0   0               3
12.464742375 2         340    Rx            8   0   0   0   0   0   0   0   0               4

Listing 7: Top rows of the plain text output result from the conversion.

In this article, we created a simple program to convert a kme50 log file to plain text. In the last article of this series, we will look at the special conditions that can arise when converting a log file.

If you have any questions, comments or suggestion for future blog articles, you can contact us directly at [email protected].


Footnotes

[1] Reading from a Kvaser Memorator and saving the logged data using kvmlib was done in www.kvaser.com/developer-blog/digging-deeper-into-kvmlib

[2] CAN FD allows data fields up to 64 bytes per frame. See the blog article www.kvaser.com/developer-blog/can-fd for more information about CAN FD.

Author Image

Magnus Carlsson

Magnus Carlsson is a Software Developer for Kvaser AB and has developed firmware and software for Kvaser products since 2007. He has also written a number of articles for Kvaser’s Developer Blog dealing with the popular Python language.

Related posts

KVASER CANlib and Python Part 3: U100 & Python Highlights
In this small guide we will explain how the LEDs on the U100 works and how to use them. Writer: Anton Carlsson,  Software Developer Cowriter: L-G Fredriksson, Field Application Engineer Version: 2021-08-09A Who should read this? This guide is meant for those who want to setup and use a Kvaser CAN interface with CANlib and...
Read more
Using CANlib Visual Studio 2017 C#.NET (canlibCLSNET)
In this document I will show you how to enable CANlib in Visual Studio 2017 when creating a C# .NET project. I will show how to create WIN32 and WIN64 applications. I will also show how to handle the platform-settings: x86, x64 and AnyCPU. When this is done, you can use the same project (source...
Read more
Using CANlib Visual Studio 2017 VB canlibCLSNET
In this document I will show you how to enable CANlib in Visual Studio 2017 when creating a Visual Basic (VB) project. I will show how to create WIN32 and WIN64 applications. I will also show how to handle the platform-settings: x86 and x64. When this is done, you can use the same project (source...
Read more
Using CANlib Visual Studio 2017 VB .NET Standard 2.0

This Dev Blog describes how to enable Kvaser’s CANlib in Visual Studio 2017 when creating a Visual Basic (VB) project (both WIN32 and WIN64 applications).

Read more
Using CANlib Visual Studio 2017 C# .NET Standard 2.0
In this document I will show you how to enable CANlib in Visual Studio 2017 when creating a C# .NET project. I will show how to create WIN32 and WIN64 applications. I will also show how to handle the platform-settings: x86, x64. When this is done, you can use the same project (source code) for...
Read more
Kvaser DIN Rail – Remote Device With I/O Pins
In this article, we take a look at the Kvaser DIN Rail and how to control this device using CANlib. Overview HW The Kvaser DIN Rail family is a collection of Ethernet to CAN/CAN FD interfaces with support for I/O ports via add-on modules. The housing has a mounting clip that attaches to a 35...
Read more
Using CANlib Visual Studio 2017 C++
In this document I will show you how to enable CANlib in Visual Studio 2017 when creating a C++ project. I will show how to create WIN32 and WIN64 applications. I will also show how to handle the platform-settings: x86 and x64. When this is done, you can use the same project (source code) for...
Read more
Running Python wrapper on Linux
This is an update of an older blog post and is now taking advantage of the Python canlib package v1.6 and later. Today we take a look at how to setup and send a CAN message using Kvasers Python package canlib. For this example we use the Kvaser USBcan Pro 2xHS v2, but any Kvaser...
Read more
Digging deeper into kvmlib
This is the third post in a 4-part series about configuring and reading logged data using a Kvaser Memorator 2nd Generation device through kvmlib: 1 Getting started with kvmlib 2 Adding script and triggers to a configuration 3 Digging deeper into kvmlib 4 Configure an SD card using kvmlib In the first part we showed...
Read more
Adding script and triggers to a configuration
This is the second post in a 4-part series about configuring and reading logged data using a Kvaser Memorator 2nd Generation device through kvmlib: 1 Getting started with kvmlib 2 Adding script and triggers to a configuration 3 Digging deeper into kvmlib 4 Configure an SD card using kvmlib In the first part we showed how...
Read more
RSSCopy Link

Get The Latest news feeds from our RSS