Re: selected rows for a collection listbox

[John DeSoi via 4D_Tech]

I found a solution that has good performance and does not require creating 
thousands of objects for rows that will never be displayed.

- The listbox collection is initially setup as a collection of row numbers. 
Using ARRAY TO COLLECTION seems to be the fastest way to do this. About 30 
milliseconds to create a collection with 500,000 rows.

- Any listbox row that is a number and not an object needs to be loaded from 
the remote server. The On Display Detail method detects this condition and 
fetches the subset of rows to be displayed. The row number in the collection 
element is replaced with the object to display. Something like:

ListBox[$row]:=New object("rowNumber";$row+1;"row";$rowObject) `Column display 
formulas based on This.row

- I can now determine the row numbers of the selected rows, even if the row has 
never been displayed. Every element of the selectedItems collection will be a 
row number or an object with a rowNumber property.

John DeSoi, Ph.D.


> On Jul 31, 2018, at 12:07 AM, Kirk Brooks via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> I think this is exactly what the new For each
> structure is meant to do - iterate over the selected elements. You don't
> need to know the indices
> if all you need is the elements. But it sounds like you want the row
> numbers to sync with the other data. In that case I don't think you have a
> choice besides an array based listbox.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Kirk Brooks via 4D_Tech]

Hey John,

On Mon, Jul 30, 2018 at 7:05 PM John DeSoi via 4D_Tech <4d_tech@lists.4d.com>
wrote:

> The selected row numbers matter if you are not using ORDA or a fully
> materialized collection of objects.

I hesitant to comment directly on what you're doing because frankly you are
frequently doing really sophisticated stuff I'm not at all sure I
understand. With that caveat I will offer that I have encountered
situations where I simply needed to do things a bit more old school -
meaning I really wanted to be able to use a collection associated with a
listbox but I just couldn't make it work. To get what I wanted I had to use
an array based listbox and some code to manage the synchronization with the
collection. It was less code than classic 4D and conceptually easier.

You've probably noticed there are some other things you can't do directly
to listboxes associated with collections:

you can't sort the listbox - but you can sort the collection
you can't add/delete rows to the listbox - you add elements to the
collection

I haven't tried dragging and dropping. You can add more columns to the
listbox which can display data based on that element of the collection. I
had a listbox/collection where each element refers to a graphic file.
Basically the contents of each element are the new Path to object command
(which is totally fabulous, BTW). I displayed the file name (This.name) and
added a column that had a method as its data source. The method made a
thumbnail of the image file and I passed the collection element as a
parameter also using 'This'. There are some redraw issues but the basic
premise works fine.

What I couldn't do was simply add a picture column and have some external
method loop through the listbox rows and populate that column with the
picture even using the same method to create it.

This may change but I think it's safe to say that if you need to need to
have that sort of row-index selection information you don't want to
associate the listbox with the collection but manage it yourself.

I have 4D 16 interfaces for Spotify and Apple Music that implemented lazy
> loading using a "fake" selection of records. Collection based list boxes
> make it possible to eliminate the table and support very powerful
> generalized code (e.g. components).
>
It is cool. I think this is what Thomas means by 'use a variable like the
database.'

I also find it strange that I can select rows with a single API call, but
> determining the selected rows takes 3 different methods depending on the
> type of list box.

True. Array based listboxes work one way, selection based ones another and
now collection based ones a third. They are three different types of data
even if managed through the same interface.

With the selected rows, I can easily iterate over the rows for any of the
> different listbox types.
>
I think this is exactly what the new For each
structure is meant to do - iterate over the selected elements. You don't
need to know the indices
if all you need is the elements. But it sounds like you want the row
numbers to sync with the other data. In that case I don't think you have a
choice besides an array based listbox.

-- 
Kirk Brooks
San Francisco, CA
===

*We go vote - they go home*
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[John DeSoi via 4D_Tech]

Hi Kirk,

Great summary, thanks for posting.


> On Jul 30, 2018, at 10:27 AM, Kirk Brooks via 4D_Tech <4d_tech@lists.4d.com> 
> wrote:
> 
> Eventually I got the concept of manipulating the data in the collection
> instead of in the listbox is the key. This is why it doesn't really matter
> which rows are selected but rather which elements are. You don't need to
> know what the row numbers are to update the associated collection you just
> need the selected items Miyako pointed to.

I promise not to post any more about it after this, but I'll say it one more 
time. The selected row numbers matter if you are not using ORDA or a fully 
materialized collection of objects. There are a lot of cool things possible 
outside of 4D data if you know the size of the result set and can load on 
demand. I have mentioned Postgres, but I'm also implementing display and 
manipulation of an ORDA selection from a remote 4D database instance over HTTPS.

I have 4D 16 interfaces for Spotify and Apple Music that implemented lazy 
loading using a "fake" selection of records. Collection based list boxes make 
it possible to eliminate the table and support very powerful generalized code 
(e.g. components).

I also find it strange that I can select rows with a single API call, but 
determining the selected rows takes 3 different methods depending on the type 
of list box. With the selected rows, I can easily iterate over the rows for any 
of the different listbox types.


John DeSoi, Ph.D.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Kirk Brooks via 4D_Tech]

I can elaborate on this a little bit - I've been working with collection
based list boxes quite a bit recently. A good starting point is the blog :

https://blog.4d.com/multilevel-collection-in-different-listboxes/
https://blog.4d.com/display-a-collection-in-a-listbox/

and Laurant's demo from the Summit.

You do not have to associate (that is, 'set the data source property to a
collection') with the Form object
. In fact lots of
the very bright folks in tech support use process variable-collections (the
first blog post does) and the magic works just fine. However, in my view
using the Form object is a tremendous feature. It allows me to make
listboxes and subforms that are task oriented and encapsulated.

The first collection based listbox I tried to create was confusing. I
expected it to work more or less like an array based one but it
doesn't. Collections
and arrays are highly analogous but they are not the same thing. For one
thing I encountered the same issue of trying to deal with the data in terms
of indices of the collection based on row numbers. One reason this doesn't
work is the row and collection index numbers don't correspond. Collections
begin counting at 0 and listboxes at 1. So listbox row #1 = collection
element 0.

Eventually I got the concept of manipulating the data in the collection
instead of in the listbox is the key. This is why it doesn't really matter
which rows are selected but rather which elements are. You don't need to
know what the row numbers are to update the associated collection you just
need the selected items Miyako pointed to.

Another place where collection and array based listboxes differ
significantly is: a collection is a reference to some underlying data while
arrays are copies of that data or the data itself. So when I display a
listbox associated with a collection I'm not duplicating the collection in
the listbox, I'm just displaying it. If I do the same thing using an array
based listbox each column in the listbox is a copy of the corresponding
data in the collection.

Visually the two could be identical but this is a huge distinction. First
it means the collection based listbox uses a lot fewer resources since it's
not copying the data. Also, the collection based listbox updates itself
when the collection changes and vice versa if the listbox is enterable.

It's hard to believe how little actual code is required to make this work
if you've done this sort of thing before - which pretty much everyone
reading this has.

Another radical difference is the associated collection can be variable
based or record based. The listbox is associated with the collection but
the collection can be built from anything. So a single listbox could
contain data from variables or records or even both concurrently.

Thomas Maul made a comment in his ORDA presentation at DC I'll paraphrase,
"You can use the database like a variable and a variable like the
database." I didn't really get it then but it resonated with me after
working with v17. ORDA is removing the hard line between record based and
variable based data in 4D.

On Mon, Jul 30, 2018 at 6:46 AM Keisuke Miyako via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> Hi,
>
> v17 introduced a new (4th) kind of list box.
>
> the coding style is quite radical compared to conventional list boxes,
> based on heavy usage of "This", "Form", and member functions.
>
> 2018/07/30 22:34、Chip Scheide via 4D_Tech <4d_tech@lists.4d.com 4d_tech@lists.4d.com>> のメール：
> what do you mean by a 'collection' listbox?
>
>
> **
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **



-- 
Kirk Brooks
San Francisco, CA
===

*We go vote - they go home*
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Chip Scheide via 4D_Tech]

Thanks
was not aware, but gathered from the continuing discussion

Chip
On Mon, 30 Jul 2018 13:45:35 +, Keisuke Miyako via 4D_Tech wrote:
> Hi,
> 
> v17 introduced a new (4th) kind of list box.
> 
> the coding style is quite radical compared to conventional list boxes,
> based on heavy usage of "This", "Form", and member functions.
> 
> 2018/07/30 22:34、Chip Scheide via 4D_Tech 
> <4d_tech@lists.4d.com> のメール：
> what do you mean by a 'collection' listbox?
> 
> 
> **
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **
---
Gas is for washing parts
Alcohol is for drinkin'
Nitromethane is for racing 
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Kirk Brooks via 4D_Tech]

Chip,
In v17 you can associate a listbox with a collection or object in addition
to the data sources we're accustomed to: arrays, current selection or named
selection. John and Miyako are talking about about one of these new ones.

On Mon, Jul 30, 2018 at 6:34 AM Chip Scheide via 4D_Tech <
4d_tech@lists.4d.com> wrote:

> John,
> what do you mean by a 'collection' listbox?
>
-- 
Kirk Brooks
San Francisco, CA
===

*We go vote - they go home*
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Keisuke Miyako via 4D_Tech]

Hi,

v17 introduced a new (4th) kind of list box.

the coding style is quite radical compared to conventional list boxes,
based on heavy usage of "This", "Form", and member functions.

2018/07/30 22:34、Chip Scheide via 4D_Tech 
<4d_tech@lists.4d.com> のメール：
what do you mean by a 'collection' listbox?


**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Chip Scheide via 4D_Tech]

John,
what do you mean by a 'collection' listbox?
if the listbox is from a selection then the highlighted record(s) are 
in the set specified by:
highlight set property
   and
the selected row(s) should be the select record number(s)

if the listbox is an array then the selected item(s) are True
while (find in array(listbox;true)>0)
  do stuff
end while


On Sun, 29 Jul 2018 10:27:43 -0500, John DeSoi via 4D_Tech wrote:
> Is there a way to get all of the highlighted row numbers for a 
> collection listbox? It seems you can only get the first highlighted 
> row number via the currentItemPosition property. The selectedItems 
> property would return a collection of highlighted items, but at best 
> you would have to sequentially search the collection to determine the 
> position for each object.
> 
> John DeSoi, Ph.D.
> 
> **
> 4D Internet Users Group (4D iNUG)
> Archive:  http://lists.4d.com/archives.html
> Options: https://lists.4d.com/mailman/options/4d_tech
> Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
> **
---
Gas is for washing parts
Alcohol is for drinkin'
Nitromethane is for racing 
**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Keisuke Miyako via 4D_Tech]

to elaborate:

since you already have ways to interrogate the remote PostgreSQL data source 
during the "On Display Detail" event,
I was thinking perhaps you could add a custom "position" property to the 
current item during the same event
for later access (I assume you only access the remote data source once, when 
the current item is null).

but I still have the feeling that completing a null-populated collection "on 
demand" is not going to be in any ways more efficient
compared to having a pre-populated placeholder collection (especially if the 
number of lines is only in the range of several thousands).

in any case, this is a very unconventional usage of collection list boxes.
my understanding is that a non-ORDA listbox is intended for usage with local 
arrays, not remote data sources.

a fully functional lazy loading system should fetch and cache several lines at 
a time and clear them to keep memory usage under control.

the call should be asynchronous to prevent the UI from blocking,
it should also cancel out redundant calls,
perhaps by using a complex system based on CALL WORKER and CALL FORM.

it's not just a question of having a "selected position collection" property.

2018/07/30 12:42、Keisuke Miyako via 4D_Tech 
<4d_tech@lists.4d.com>のメール:

but I respect your decision.

have you tried the "Displayed line number" function in your "On Display Detail" 
code?

http://doc.4d.com/4Dv16/4D/16.3/Displayed-line-number.301-3652452.en.html



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Keisuke Miyako via 4D_Tech]

well that's a shame...
it sound like a self-imposed design restriction,
I fail to see how it can deliver any substantial benefits.

but I respect your decision.

have you tried the "Displayed line number" function in your "On Display Detail" 
code?

http://doc.4d.com/4Dv16/4D/16.3/Displayed-line-number.301-3652452.en.html

2018/07/30 12:34、John DeSoi via 4D_Tech 
<4d_tech@lists.4d.com>のメール:

I want to leave the collection elements as Null and only load them on demand 
(displayed or the user selected them to do something). I'm trying to avoid 
creating objects for thousands of rows that will never be displayed or used.


**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[John DeSoi via 4D_Tech]

> On Jul 29, 2018, at 9:41 PM, Keisuke Miyako via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> the beauty of working with collections (in ORDA) is that
> when you operate via member methods or "For each",
> the elements are loaded on demand.
> there is no need to work with the "On Display Detail" event.

Yes, nice model and works great if you are using ORDA. I'm not using ORDA here 
and want to take advantage of the same type of on demand loading. 4D thought it 
was important enough to tell you the row number of the selected collection 
item, but only the first one, not all of them. We have LISTBOX SELECT ROW which 
can work with any kind of listbox, but to get the selected rows there are 3 
different methods depending on what type of listbox is used. Would be nice to 
have LISTBOX SELECTED ROWS(array or collection) that would work on any listbox 
type. 

> how exactly did you populate the collection that you display in the listbox?
> it sounds like you have a place holder object for each line.

Yes.

> 
> why not use the "map" member function to add a custom "position (offset)" 
> property?

I want to leave the collection elements as Null and only load them on demand 
(displayed or the user selected them to do something). I'm trying to avoid 
creating objects for thousands of rows that will never be displayed or used.


John DeSoi, Ph.D.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Keisuke Miyako via 4D_Tech]

normally with collections, you shouldn't have to care about the position of 
each element.
it's how "For each" works and member functions such as "map" or "filter" works.

"to operate on the selected rows as a collection implies that every row has to 
be loaded"

collection data source of a list box can be an entity selection (ORDA) or a 
local collection (array of objects).

the former is comparable to a (named) selection listbox, with lazy loading and 
all,
but the latter is basically a generic version of the classic array-based list 
box.

so yes, I suppose you're right, if you are not using ORDA.

but that is not an innate constraint of the collection type.
rather, it is typical of an object that uses an array as its data source.

the beauty of working with collections (in ORDA) is that
when you operate via member methods or "For each",
the elements are loaded on demand.
there is no need to work with the "On Display Detail" event.

---

how exactly did you populate the collection that you display in the listbox?
it sounds like you have a place holder object for each line.

why not use the "map" member function to add a custom "position (offset)" 
property?

2018/07/30 11:20、John DeSoi via 4D_Tech 
<4d_tech@lists.4d.com>のメール:

It is not ORDA. I'm displaying a server side cursor from PostgreSQL. The select 
result may have hundreds of thousands of rows. I can scroll that nicely in 4D 
using On Display Detail to load only the visible data. But to operate on the 
selected rows as a collection implies that every row has to be loaded. If I 
could get just the selected *row numbers*, I could load those on demand.



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[John DeSoi via 4D_Tech]

It is not ORDA. I'm displaying a server side cursor from PostgreSQL. The select 
result may have hundreds of thousands of rows. I can scroll that nicely in 4D 
using On Display Detail to load only the visible data. But to operate on the 
selected rows as a collection implies that every row has to be loaded. If I 
could get just the selected *row numbers*, I could load those on demand.

John DeSoi, Ph.D.
 



> On Jul 29, 2018, at 8:47 PM, Keisuke Miyako via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> a collection of objects rather than the row numbers makes this impractical.
> 
> impractical how?
> 
> ORDA is lazy loading by nature, so it doesn't matter if the row is visible or 
> not.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Keisuke Miyako via 4D_Tech]

a collection of objects rather than the row numbers makes this impractical.

impractical how?

ORDA is lazy loading by nature, so it doesn't matter if the row is visible or 
not.

2018/07/30 10:13、John DeSoi via 4D_Tech 
<4d_tech@lists.4d.com>のメール:

How does the third one give me the *row numbers* of the highlighted rows other 
than sequentially searching for each item? That was my original question.

I want to use a collection based list box to represent data from a remote 
source. I only need to load enough data to display the visible rows. But the 
selection model that returns a collection of objects rather than the row 
numbers makes this impractical.



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[John DeSoi via 4D_Tech]

How does the third one give me the *row numbers* of the highlighted rows other 
than sequentially searching for each item? That was my original question.

I want to use a collection based list box to represent data from a remote 
source. I only need to load enough data to display the visible rows. But the 
selection model that returns a collection of objects rather than the row 
numbers makes this impractical. 

I'll make a feature request.

Thanks,

John DeSoi, Ph.D.



> On Jul 29, 2018, at 7:28 PM, Keisuke Miyako via 4D_Tech 
> <4d_tech@lists.4d.com> wrote:
> 
> there are 3 properties you can have 4D manage automatically (that is, 
> read-only by code)
> 
> current item (typically an object)
> current item position (typically numeric), and
> selected items (typically a collection)
> 
> http://doc.4d.com/4Dv17/4D/17/List-box-specific-properties.300-3743666.en.html
> 
> use the 3rd one.
> 
> 2018/07/30 0:27、John DeSoi via 4D_Tech 
> <4d_tech@lists.4d.com>のメール:
> 
> Is there a way to get all of the highlighted row numbers for a collection 
> listbox? It seems you can only get the first highlighted row number via the 
> currentItemPosition property. The selectedItems property would return a 
> collection of highlighted items, but at best you would have to sequentially 
> search the collection to determine the position for each object.

**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**

Re: selected rows for a collection listbox

[Keisuke Miyako via 4D_Tech]

there are 3 properties you can have 4D manage automatically (that is, read-only 
by code)

current item (typically an object)
current item position (typically numeric), and
selected items (typically a collection)

http://doc.4d.com/4Dv17/4D/17/List-box-specific-properties.300-3743666.en.html

use the 3rd one.

2018/07/30 0:27、John DeSoi via 4D_Tech 
<4d_tech@lists.4d.com>のメール:

Is there a way to get all of the highlighted row numbers for a collection 
listbox? It seems you can only get the first highlighted row number via the 
currentItemPosition property. The selectedItems property would return a 
collection of highlighted items, but at best you would have to sequentially 
search the collection to determine the position for each object.



**
4D Internet Users Group (4D iNUG)
Archive:  http://lists.4d.com/archives.html
Options: https://lists.4d.com/mailman/options/4d_tech
Unsub:  mailto:4d_tech-unsubscr...@lists.4d.com
**