Is there an alternative to the very slow MemoLine() function?

I Love Xbase++ (ILX)
The portal for Xbase++ developers worldwide

Andreas Herdt

Member
I am here to help you!
Feb 1, 2021
31
4
8
Customer Identifier
E104394

Background​

The functions MLCount() and MemoLine() offer a very simple way to extract lines from a string which are separated by hard returns (Chr(13)+Chr(10)). The performance of the function MemoLine() breaks down very much with strings containing many lines. The reason is that this function is stateless and therefore each call the desired line is identified from scratch in the string.

Example​

There is no fast alternative available for the MemoLine() function. However, with a few lines of code a better solution can be found. In the following example, we assume that all lines are to extracted from a string containing CRLFs as the line delimiter. Please check the parameters you pass to the MemoLine() function in your code to see how you use that function in respect of tab size, line length and wrapping. The code in this example may need to be extended to meet your requirements.

Furthermore, the use of the alternative proposed here should be simpler than using MemoLine() and follow the following coding pattern:
Xbase++:
   ...
   oLineIterator := LineIterator():new( cInput )
   DO WHILE .NOT. oLineIterator:isEnd()

      cLine := oLineIterator:getLine()

      ? cLine

      oLineIterator:skip()

   ENDDO
   ...
The LineIterator class used above with the required :isEnd(), :getLine() and :skip() methods could be implemented as follows:
Xbase++:
#define CRLF Chr(13) + Chr(10)
/// <summary>
/// The class LineIterator is suitable to extract line from a
/// character string that are separated by hard return. The string
/// separator is CRLF by default can be customized by the second
/// optional :init() parameter
/// </summary>
CLASS LineIterator
   PROTECTED:
      VAR _Data
      VAR _Separator
      VAR _DataLen
      VAR _SepLen
      VAR _CurrPos
      VAR _SepPos
      VAR _Trim
   EXPORTED:
      METHOD init
      METHOD suppressTrim
      METHOD reset
      METHOD isEnd
      METHOD skip
      METHOD getLine
ENDCLASS

/// <summary>
/// Initialize the Object
/// <param name="cText">The character string to be scanned.</param>
/// <param name="cSeparator">The optional separator is CRLF by default</param>
/// </summary>
METHOD LineIterator:init( cText, cSeparator )
   ::_Trim := .T.
   ::_Data := cText
   ::_Separator := Coalesce( cSeparator, CRLF )
   ::reset()
RETURN SELF

/// <summary>
/// Call this method to suppress the AllTrim() operation on
/// of the line
/// </summary>
METHOD LineIterator:suppressTrim()
   ::_Trim := .F.
RETURN SELF

/// <summary>
/// Resets the iterator
/// </summary>
METHOD LineIterator:reset()
   ::_CurrPos := 1
   ::_SepPos := At( ::_Separator, ::_Data, ::_CurrPos )
   ::_DataLen := Len( ::_Data )
   ::_SepLen  := Len( ::_Separator )
RETURN SELF

/// <summary>
/// Verify whether there is more data
/// </summary>
METHOD LineIterator:isEnd()
RETURN ::_CurrPos > ::_DataLen

/// <summary>
/// Go to next data junk
/// </summary>
METHOD LineIterator:skip()

   IF ::isEnd()
      RETURN self
   ENDIF

   ::_CurrPos := ::_SepPos + ::_SepLen
   ::_SepPos := At( ::_Separator, ::_Data, ::_SepPos + ::_SepLen )

RETURN SELF

/// <summary>
/// Extract a line.
/// </summary>
METHOD LineIterator:getLine()
   LOCAL cResult

   IF ::_SepPos == 0
      cResult := SubStr( ::_Data, ::_CurrPos )
      // Input does not terminate with separator so go beyond end of input here
      ::_CurrPos := ::_DataLen + 1
   ELSE
      cResult := SubStr( ::_Data, ::_CurrPos, ::_SepPos - ::_CurrPos )
   ENDIF

   IF ::_Trim
      cResult := AllTrim( cResult )
   ENDIF

RETURN cResult
Objects of the above LineIterator class know the current position in the string and can therefore determine the next line very quickly.

Because the separator is parameterized in the :init() method, the class is also suitable for extracting comma-separated values from a line.
 
Last edited: