Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-21 Thread Andy Shevchenko
On Tue, Jan 21, 2020 at 2:39 AM AKASHI Takahiro
 wrote:
> On Fri, Jan 17, 2020 at 11:47:03AM +0200, Andy Shevchenko wrote:
> > On Fri, Jan 17, 2020 at 8:13 AM AKASHI Takahiro
> >  wrote:

...

> Really? I've got messed up now.
> - Why do you need to use g_multi to access the disk?

How else you may provide a way to access it from remote host via USB?
Actually it's not me it's a stock image on that board. But it doesn't matter.

> - Where is a disk attached to?

To my desktop system.
Or you are asked how it's connected to the embedded board? It's eMMC
chip soldered on it.

> - Are Linux and Windows running on the same board?

No, it's not the case.

[ embedded board + U-Boot + stock OS] ---USB--- [Windows 10 desktop]
  |
  v
[ eMMC soldered on embedded board, *partition* is shared as a disk]

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-20 Thread AKASHI Takahiro
On Fri, Jan 17, 2020 at 11:47:03AM +0200, Andy Shevchenko wrote:
> On Fri, Jan 17, 2020 at 8:13 AM AKASHI Takahiro
>  wrote:
> > On Thu, Jan 16, 2020 at 10:31:49PM +0200, Andy Shevchenko wrote:
> 
> ...
> 
> > > Prerequisites:
> > >  - the board with U-Boot and installed Linux OS on eMMC
> > >  - g_multi module in Linux OS that shares *one of the eMMC partitions*
> > > (pay attention here) as a disk to Windows host
> >
> > I also misunderstood your assumption above; You are developing
> > a linux-based USB gadget for Windows (10)?
> 
> No.

Really? I've got messed up now.
- Why do you need to use g_multi to access the disk?
- Where is a disk attached to?
- Are Linux and Windows running on the same board?

-Takahiro Akashi

> > > Now, when you format that exposed disk (which is actually a partition
> > > on eMMC!) in Windows, you will get nested partitioning.
> >
> > So why do you want to access *that* partition from U-Boot on the board?
> > I don't think it is a common case.
> 
> What I'm trying to do is to copy some bootables to that partition from
> Windows machine in order to boot them via U-Boot.
> That's allow me not to disturb stock image on that board.
> 
> If U-Boot is not going to support nested partitioning, perhaps I can
> submit a documentation fix to advertise this explicitly.
> 
> -- 
> With Best Regards,
> Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-17 Thread Tom Rini
On Fri, Jan 17, 2020 at 11:47:03AM +0200, Andy Shevchenko wrote:
> On Fri, Jan 17, 2020 at 8:13 AM AKASHI Takahiro
>  wrote:
> > On Thu, Jan 16, 2020 at 10:31:49PM +0200, Andy Shevchenko wrote:
> 
> ...
> 
> > > Prerequisites:
> > >  - the board with U-Boot and installed Linux OS on eMMC
> > >  - g_multi module in Linux OS that shares *one of the eMMC partitions*
> > > (pay attention here) as a disk to Windows host
> >
> > I also misunderstood your assumption above; You are developing
> > a linux-based USB gadget for Windows (10)?
> 
> No.
> 
> > > Now, when you format that exposed disk (which is actually a partition
> > > on eMMC!) in Windows, you will get nested partitioning.
> >
> > So why do you want to access *that* partition from U-Boot on the board?
> > I don't think it is a common case.
> 
> What I'm trying to do is to copy some bootables to that partition from
> Windows machine in order to boot them via U-Boot.
> That's allow me not to disturb stock image on that board.
> 
> If U-Boot is not going to support nested partitioning, perhaps I can
> submit a documentation fix to advertise this explicitly.

I'm confused here, sorry.  I thought one of the described examples was
how to have Windows use the whole device it was given, without partition
table, as FAT and thus it would be shareable.  Is that not possible?

-- 
Tom


signature.asc
Description: PGP signature


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-17 Thread Andy Shevchenko
On Fri, Jan 17, 2020 at 8:13 AM AKASHI Takahiro
 wrote:
> On Thu, Jan 16, 2020 at 10:31:49PM +0200, Andy Shevchenko wrote:

...

> > Prerequisites:
> >  - the board with U-Boot and installed Linux OS on eMMC
> >  - g_multi module in Linux OS that shares *one of the eMMC partitions*
> > (pay attention here) as a disk to Windows host
>
> I also misunderstood your assumption above; You are developing
> a linux-based USB gadget for Windows (10)?

No.

> > Now, when you format that exposed disk (which is actually a partition
> > on eMMC!) in Windows, you will get nested partitioning.
>
> So why do you want to access *that* partition from U-Boot on the board?
> I don't think it is a common case.

What I'm trying to do is to copy some bootables to that partition from
Windows machine in order to boot them via U-Boot.
That's allow me not to disturb stock image on that board.

If U-Boot is not going to support nested partitioning, perhaps I can
submit a documentation fix to advertise this explicitly.

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-16 Thread AKASHI Takahiro
On Thu, Jan 16, 2020 at 10:31:49PM +0200, Andy Shevchenko wrote:
> On Thu, Jan 16, 2020 at 9:20 PM Heinrich Schuchardt  
> wrote:
> > On 1/16/20 11:39 AM, Andy Shevchenko wrote:
> > >
> > >>> Obviously U-Boot's fat code cannot handle it.
> > >>
> > >> So precisely, U-Boot cannot handle nested partition( table)s?
> > >
> > > Seems so. We need to be able to supply the partition number we would
> > > like to open, something like
> > >cmd  [[:[:]]]
> > > otherwise it will require some (error prone) heuristics to understand
> > > which one user would like to use.
> > >
> >
> > I have formatted a USB stick Windows 10.
> 
> That is *not* the case I'm describing.
> 
> Maybe I described it wrong. Let me try again.
> 
> Prerequisites:
>  - the board with U-Boot and installed Linux OS on eMMC
>  - g_multi module in Linux OS that shares *one of the eMMC partitions*
> (pay attention here) as a disk to Windows host

I also misunderstood your assumption above; You are developing
a linux-based USB gadget for Windows (10)?

> Now, when you format that exposed disk (which is actually a partition
> on eMMC!) in Windows, you will get nested partitioning.

So why do you want to access *that* partition from U-Boot on the board?
I don't think it is a common case.

-Takahiro Akashi

> P.S. I can easily reproduce this on real device with latest U-Boot.
> U-Boot has obvious issue with recognizing such disks.
> 
> -- 
> With Best Regards,
> Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-16 Thread Andy Shevchenko
On Thu, Jan 16, 2020 at 9:20 PM Heinrich Schuchardt  wrote:
> On 1/16/20 11:39 AM, Andy Shevchenko wrote:
> >
> >>> Obviously U-Boot's fat code cannot handle it.
> >>
> >> So precisely, U-Boot cannot handle nested partition( table)s?
> >
> > Seems so. We need to be able to supply the partition number we would
> > like to open, something like
> >cmd  [[:[:]]]
> > otherwise it will require some (error prone) heuristics to understand
> > which one user would like to use.
> >
>
> I have formatted a USB stick Windows 10.

That is *not* the case I'm describing.

Maybe I described it wrong. Let me try again.

Prerequisites:
 - the board with U-Boot and installed Linux OS on eMMC
 - g_multi module in Linux OS that shares *one of the eMMC partitions*
(pay attention here) as a disk to Windows host

Now, when you format that exposed disk (which is actually a partition
on eMMC!) in Windows, you will get nested partitioning.

P.S. I can easily reproduce this on real device with latest U-Boot.
U-Boot has obvious issue with recognizing such disks.

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-16 Thread Heinrich Schuchardt

On 1/16/20 11:39 AM, Andy Shevchenko wrote:



Obviously U-Boot's fat code cannot handle it.


So precisely, U-Boot cannot handle nested partition( table)s?


Seems so. We need to be able to supply the partition number we would
like to open, something like
   cmd  [[:[:]]]
otherwise it will require some (error prone) heuristics to understand
which one user would like to use.



I have formatted a USB stick Windows 10.

When I display the partition table with fdisk it shows:

Device Boot  StartEndSectors   Size Id Type
/dev/sdb1   1935758368 3615603091 1679844724   801G 61 SpeedStor
/dev/sdb20  0  0 0B 65 Novell
Netware 386
/dev/sdb4 28049408   28049850443 221.5K  0 Empty

But there is no partition table at all. The FAT filesystem starts at
sector 0. I can mount the disk with

sudo mount /dev/sdb /mnt

In U-Boot I can read the file system as partition 0:

=> ls scsi 0:0
System Volume Information/
   11   test.txt.txt

1 file(s), 1 dir(s)

=> load scsi 0:0 0x4020 test.txt.txt
11 bytes read in 10 ms (1000 Bytes/s)
=> md.b 0x4020 b
4020: 48 65 6c 6c 6f 20 77 6f 72 6c 64   Hello world
=>

The UEFI shell started from U-Boot sees the file system:

Mapping table
  FS0: Alias(s):F0a0:;BLK0:
  /VenHw(e61d73b9-a384-4acc-aeab-82e828f3628b)/Scsi(0,0)

In the UEFI shell I created a new text file. I could read it in Windows.

Now I formatted the USB in Linux with gdisk to give it a GPT partition
table:

Number  Start (sector)End (sector)  Size   Code  Name
   1204833556479   16.0 GiB0700  Microsoft
basic data
   23355648062521310   13.8 GiB0700  Microsoft
basic data

and formatted the partitions with mkfs.vfat as FAT32.

Now Windows sees this USB stick as two partions and can read files
created in Linux.

So this is what I learnt:

Windows 10 does not support MBR partition tables on USB sticks.
Windows 10 supports GPT partition tables on USB sticks.
Windows formats empty USB sticks without any partition table at all.
There is no obvious compatibility issue in U-Boot.

Best regards

Heinrich


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-16 Thread Andy Shevchenko
On Thu, Jan 16, 2020 at 4:01 AM AKASHI Takahiro
 wrote:
> On Wed, Jan 15, 2020 at 09:12:59AM +0900, AKASHI Takahiro wrote:
> > On Tue, Jan 14, 2020 at 02:43:43PM +0200, Andy Shevchenko wrote:
> > > On Tue, Jan 14, 2020 at 10:23 AM Andy Shevchenko
> > >  wrote:
> > > > On Tue, Jan 14, 2020 at 10:21 AM Andy Shevchenko
> > > >  wrote:
> > > > > On Tue, Jan 14, 2020 at 1:14 AM Heinrich Schuchardt 
> > > > >  wrote:
> > > > > > On 1/13/20 10:52 PM, Andy Shevchenko wrote:
> > > >
> > > > ...
> > > >
> > > > > > This image loads fine on current U-Boot, see below.
> > > > >
> > > > > Of course it does *in the test case you have done*.
> > > > > I'm describing different one. The provided image must be a *partition*
> > > > > on the real disk.
> > > > >
> > > > > So, before use it the preparatory steps must be made.
> > > > >
> > > > > Something like
> > > > >
> > > > > % dd if=/dev/zero of=image-file bs=1M count=1000
> > > > > % fdisk image-file
> > > > >  ...create a partition table, where one partition has a (similar) size
> > > > > of the image I provided
> > > > > % mount -o loop,offset=... image-file /mnt # use *partition* as a 
> > > > > disk!
> > > > > % dd --sparse if=mmc-fat-part of=/mnt
> > > > > % umount /mnt
> > > > >
> > > > > And use image-file instead.
> > > >
> > > > Should I prepare it for you or you can do it yourself?
> > >
> > > It's there under name image-file.gz
> >
> > ===8<===
> > $ hd image-file
> >   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  
> > ||
> > *
> > 01b0  00 00 00 00 00 00 00 00  59 02 8c 89 00 00 00 20  
> > |Y.. |
> > 01c0  21 00 0c 08 27 62 00 08  00 00 01 00 18 00 00 00  
> > |!...'b..|
> > 01d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  
> > ||
> > *
> > (snip)
> > 0010  eb 3c 90 6d 6b 64 6f 73  66 73 00 00 02 20 01 00  |.<.mkdosfs... 
> > ..|
> > 00100010  02 00 02 00 00 f8 c0 00  10 00 04 00 00 00 00 00  
> > ||
> > 00100020  00 00 18 00 80 00 29 ea  36 23 57 20 20 20 20 20  |..).6#W
> >  |
> > 00100030  20 20 20 20 20 20 46 41  54 31 36 20 20 20 0e 1f  |  FAT16   
> > ..|
> > (snip)
> > 001001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  
> > ||
> > *
> > 001001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  
> > |..U.|
> > 00100200  f8 ff ff ff 00 00 00 00  00 00 00 00 00 00 00 00  
> > ||
> > 00100210  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  
> > ||
> > *
> > 0011  eb 58 90 4d 53 44 4f 53  35 2e 30 00 02 08 1a 14  
> > |.X.MSDOS5.0.|
> > 00110010  02 00 00 00 00 f8 00 00  3f 00 ff 00 80 00 00 00  
> > |?...|
> > 00110020  00 e8 17 00 f3 05 00 00  00 00 00 00 02 00 00 00  
> > ||
> > 00110030  01 00 06 00 00 00 00 00  00 00 00 00 00 00 00 00  
> > ||
> > 00110040  80 00 29 f8 a9 74 d0 4e  4f 20 4e 41 4d 45 20 20  |..)..t.NO NAME 
> >  |
> > 00110050  20 20 46 41 54 33 32 20  20 20 33 c9 8e d1 bc f4  |  FAT32   
> > 3.|
> > (snip)
> > ===>8===
> >
> > [0x10-0x100200) looks to be PBR.
> > [0x11-0x110050) looks to be MBR.
> > But I don't know what is [0x0-0x10).
>
> (Correction)
> [0x0-0x200) is actually a partition table:
>
>   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 01b0  00 00 00 00 00 00 00 00  59 02 8c 89 00 00/00/20  |Y.. |
>  ^^:boot_ind
> boot_ind: not ACTIVE
> 01c0  21 00 0c 08 27 62/00 08  00 00/01 00 18 00/00 00  |!...'b..|
> p1's start   p1's size
> start: 0x0800 sector (= 0x10)
> size:  0x0018 sectors
> 01d0  00 00/00 00 00 00/00 00  00 00/00 00 00 00/00 00  ||
> *
> 01f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..U.|
>  ^:magic number
>
> And [0x10-0x100200) is a PBR, which then points to a next-level
> partition:
>
> 0010  eb 3c 90 6d 6b 64 6f 73  66 73 00 00 02 20 01 00  |.<.mkdosfs... ..|
> (snip)
> 001001b0  00 00 00 00 00 00 00 00  8c 60 88 d5 00 00/00/02/ |.`..|
> 001001c0  03 00 0c fe 3f 60/80 00  00 00/00 e8 17 00/00 00  |?`..|
> startsize
> start: 0x0080
> size:  0x0017e800

Thank you for detailed explanation.

> > Obviously U-Boot's fat code cannot handle it.
>
> So precisely, U-Boot cannot handle nested partition( table)s?

Seems so. We need to be able to supply the partition number we would
like to open, something like
  cmd  [[:[:]]]
otherwise it will require some (error prone) heuristics to understand
which one user would like to use.

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-15 Thread AKASHI Takahiro
On Wed, Jan 15, 2020 at 09:12:59AM +0900, AKASHI Takahiro wrote:
> On Tue, Jan 14, 2020 at 02:43:43PM +0200, Andy Shevchenko wrote:
> > On Tue, Jan 14, 2020 at 10:23 AM Andy Shevchenko
> >  wrote:
> > > On Tue, Jan 14, 2020 at 10:21 AM Andy Shevchenko
> > >  wrote:
> > > > On Tue, Jan 14, 2020 at 1:14 AM Heinrich Schuchardt 
> > > >  wrote:
> > > > > On 1/13/20 10:52 PM, Andy Shevchenko wrote:
> > >
> > > ...
> > >
> > > > > This image loads fine on current U-Boot, see below.
> > > >
> > > > Of course it does *in the test case you have done*.
> > > > I'm describing different one. The provided image must be a *partition*
> > > > on the real disk.
> > > >
> > > > So, before use it the preparatory steps must be made.
> > > >
> > > > Something like
> > > >
> > > > % dd if=/dev/zero of=image-file bs=1M count=1000
> > > > % fdisk image-file
> > > >  ...create a partition table, where one partition has a (similar) size
> > > > of the image I provided
> > > > % mount -o loop,offset=... image-file /mnt # use *partition* as a disk!
> > > > % dd --sparse if=mmc-fat-part of=/mnt
> > > > % umount /mnt
> > > >
> > > > And use image-file instead.
> > >
> > > Should I prepare it for you or you can do it yourself?
> > 
> > It's there under name image-file.gz
> 
> ===8<===
> $ hd image-file
>   00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 01b0  00 00 00 00 00 00 00 00  59 02 8c 89 00 00 00 20  |Y.. |
> 01c0  21 00 0c 08 27 62 00 08  00 00 01 00 18 00 00 00  |!...'b..|
> 01d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> (snip)
> 0010  eb 3c 90 6d 6b 64 6f 73  66 73 00 00 02 20 01 00  |.<.mkdosfs... ..|
> 00100010  02 00 02 00 00 f8 c0 00  10 00 04 00 00 00 00 00  ||
> 00100020  00 00 18 00 80 00 29 ea  36 23 57 20 20 20 20 20  |..).6#W |
> 00100030  20 20 20 20 20 20 46 41  54 31 36 20 20 20 0e 1f  |  FAT16   ..|
> (snip)
> 001001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 001001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..U.|
> 00100200  f8 ff ff ff 00 00 00 00  00 00 00 00 00 00 00 00  ||
> 00100210  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> *
> 0011  eb 58 90 4d 53 44 4f 53  35 2e 30 00 02 08 1a 14  |.X.MSDOS5.0.|
> 00110010  02 00 00 00 00 f8 00 00  3f 00 ff 00 80 00 00 00  |?...|
> 00110020  00 e8 17 00 f3 05 00 00  00 00 00 00 02 00 00 00  ||
> 00110030  01 00 06 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
> 00110040  80 00 29 f8 a9 74 d0 4e  4f 20 4e 41 4d 45 20 20  |..)..t.NO NAME  |
> 00110050  20 20 46 41 54 33 32 20  20 20 33 c9 8e d1 bc f4  |  FAT32   3.|
> (snip)
> ===>8===
> 
> [0x10-0x100200) looks to be PBR.
> [0x11-0x110050) looks to be MBR.
> But I don't know what is [0x0-0x10).

(Correction)
[0x0-0x200) is actually a partition table:

  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
*
01b0  00 00 00 00 00 00 00 00  59 02 8c 89 00 00/00/20  |Y.. |
 ^^:boot_ind
boot_ind: not ACTIVE
01c0  21 00 0c 08 27 62/00 08  00 00/01 00 18 00/00 00  |!...'b..|
p1's start   p1's size
start: 0x0800 sector (= 0x10)
size:  0x0018 sectors
01d0  00 00/00 00 00 00/00 00  00 00/00 00 00 00/00 00  ||
*
01f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..U.|
 ^:magic number

And [0x10-0x100200) is a PBR, which then points to a next-level
partition:

0010  eb 3c 90 6d 6b 64 6f 73  66 73 00 00 02 20 01 00  |.<.mkdosfs... ..|
(snip)
001001b0  00 00 00 00 00 00 00 00  8c 60 88 d5 00 00/00/02/ |.`..|
001001c0  03 00 0c fe 3f 60/80 00  00 00/00 e8 17 00/00 00  |?`..|
startsize
start: 0x0080
size:  0x0017e800

> Obviously U-Boot's fat code cannot handle it.

So precisely, U-Boot cannot handle nested partition( table)s?

-Takahiro Akashi

> 
> Thanks,
> -Takahiro Akashi
> 
> 
> > Commit messages have the commands I performed to get this image cooked.
> > 
> > -- 
> > With Best Regards,
> > Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-14 Thread AKASHI Takahiro
On Tue, Jan 14, 2020 at 02:43:43PM +0200, Andy Shevchenko wrote:
> On Tue, Jan 14, 2020 at 10:23 AM Andy Shevchenko
>  wrote:
> > On Tue, Jan 14, 2020 at 10:21 AM Andy Shevchenko
> >  wrote:
> > > On Tue, Jan 14, 2020 at 1:14 AM Heinrich Schuchardt  
> > > wrote:
> > > > On 1/13/20 10:52 PM, Andy Shevchenko wrote:
> >
> > ...
> >
> > > > This image loads fine on current U-Boot, see below.
> > >
> > > Of course it does *in the test case you have done*.
> > > I'm describing different one. The provided image must be a *partition*
> > > on the real disk.
> > >
> > > So, before use it the preparatory steps must be made.
> > >
> > > Something like
> > >
> > > % dd if=/dev/zero of=image-file bs=1M count=1000
> > > % fdisk image-file
> > >  ...create a partition table, where one partition has a (similar) size
> > > of the image I provided
> > > % mount -o loop,offset=... image-file /mnt # use *partition* as a disk!
> > > % dd --sparse if=mmc-fat-part of=/mnt
> > > % umount /mnt
> > >
> > > And use image-file instead.
> >
> > Should I prepare it for you or you can do it yourself?
> 
> It's there under name image-file.gz

===8<===
$ hd image-file
  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
*
01b0  00 00 00 00 00 00 00 00  59 02 8c 89 00 00 00 20  |Y.. |
01c0  21 00 0c 08 27 62 00 08  00 00 01 00 18 00 00 00  |!...'b..|
01d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
*
(snip)
0010  eb 3c 90 6d 6b 64 6f 73  66 73 00 00 02 20 01 00  |.<.mkdosfs... ..|
00100010  02 00 02 00 00 f8 c0 00  10 00 04 00 00 00 00 00  ||
00100020  00 00 18 00 80 00 29 ea  36 23 57 20 20 20 20 20  |..).6#W |
00100030  20 20 20 20 20 20 46 41  54 31 36 20 20 20 0e 1f  |  FAT16   ..|
(snip)
001001d0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
*
001001f0  00 00 00 00 00 00 00 00  00 00 00 00 00 00 55 aa  |..U.|
00100200  f8 ff ff ff 00 00 00 00  00 00 00 00 00 00 00 00  ||
00100210  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
*
0011  eb 58 90 4d 53 44 4f 53  35 2e 30 00 02 08 1a 14  |.X.MSDOS5.0.|
00110010  02 00 00 00 00 f8 00 00  3f 00 ff 00 80 00 00 00  |?...|
00110020  00 e8 17 00 f3 05 00 00  00 00 00 00 02 00 00 00  ||
00110030  01 00 06 00 00 00 00 00  00 00 00 00 00 00 00 00  ||
00110040  80 00 29 f8 a9 74 d0 4e  4f 20 4e 41 4d 45 20 20  |..)..t.NO NAME  |
00110050  20 20 46 41 54 33 32 20  20 20 33 c9 8e d1 bc f4  |  FAT32   3.|
(snip)
===>8===

[0x10-0x100200) looks to be PBR.
[0x11-0x110050) looks to be MBR.
But I don't know what is [0x0-0x10).
Obviously U-Boot's fat code cannot handle it.

Thanks,
-Takahiro Akashi


> Commit messages have the commands I performed to get this image cooked.
> 
> -- 
> With Best Regards,
> Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-14 Thread Andy Shevchenko
On Tue, Jan 14, 2020 at 2:43 PM Andy Shevchenko
 wrote:

$ qemu-system-i386 -m 1G -bios u-boot.rom -nographic -drive
if=none,file=/tmp/for-
andy/image-file,format=raw,id=mydisk -no-reboot -device
ich9-ahci,id=ahci -device ide-drive,drive=mydisk,bus=ahci.
0
qemu-system-i386: -device ide-drive,drive=mydisk,bus=ahci.0: warning:
'ide-drive' is deprecated, please use 'ide-h
d' or 'ide-cd' instead


U-Boot 2020.01-00012-gb4daeb7f02 (Jan 14 2020 - 15:06:52 +0200)

CPU:   QEMU Virtual CPU version 2.5+
DRAM:  1 GiB
Video: 1024x768x32
Model: QEMU x86 (I440FX)
Net:   e1000: 52:54:00:12:34:56

Warning: e1000#0 using MAC address from ROM
eth0: e1000#0
Hit any key to stop autoboot:  0
=> scsi scan
scanning bus for devices...
Target spinup took 0 ms.
SATA link 1 timeout.
SATA link 2 timeout.
SATA link 3 timeout.
SATA link 4 timeout.
SATA link 5 timeout.
AHCI 0001. 32 slots 6 ports 1.5 Gbps 0x3f impl SATA mode
flags: 64bit ncq only
 Device 0: (0:0) Vendor: ATA Prod.: QEMU HARDDISK Rev: 2.5+
   Type: Hard Disk
   Capacity: 1024.0 MB = 1.0 GB (2097152 x 512)
=> ls scsi 0:0
** Unrecognized filesystem type **
=> ls scsi 0:1

0 file(s), 0 dir(s)

=> ls scsi 0:2
** Invalid partition 2 **
=>

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-14 Thread Andy Shevchenko
On Tue, Jan 14, 2020 at 10:23 AM Andy Shevchenko
 wrote:
> On Tue, Jan 14, 2020 at 10:21 AM Andy Shevchenko
>  wrote:
> > On Tue, Jan 14, 2020 at 1:14 AM Heinrich Schuchardt  
> > wrote:
> > > On 1/13/20 10:52 PM, Andy Shevchenko wrote:
>
> ...
>
> > > This image loads fine on current U-Boot, see below.
> >
> > Of course it does *in the test case you have done*.
> > I'm describing different one. The provided image must be a *partition*
> > on the real disk.
> >
> > So, before use it the preparatory steps must be made.
> >
> > Something like
> >
> > % dd if=/dev/zero of=image-file bs=1M count=1000
> > % fdisk image-file
> >  ...create a partition table, where one partition has a (similar) size
> > of the image I provided
> > % mount -o loop,offset=... image-file /mnt # use *partition* as a disk!
> > % dd --sparse if=mmc-fat-part of=/mnt
> > % umount /mnt
> >
> > And use image-file instead.
>
> Should I prepare it for you or you can do it yourself?

It's there under name image-file.gz

Commit messages have the commands I performed to get this image cooked.

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-14 Thread Andy Shevchenko
On Tue, Jan 14, 2020 at 10:21 AM Andy Shevchenko
 wrote:
> On Tue, Jan 14, 2020 at 1:14 AM Heinrich Schuchardt  
> wrote:
> > On 1/13/20 10:52 PM, Andy Shevchenko wrote:

...

> > This image loads fine on current U-Boot, see below.
>
> Of course it does *in the test case you have done*.
> I'm describing different one. The provided image must be a *partition*
> on the real disk.
>
> So, before use it the preparatory steps must be made.
>
> Something like
>
> % dd if=/dev/zero of=image-file bs=1M count=1000
> % fdisk image-file
>  ...create a partition table, where one partition has a (similar) size
> of the image I provided
> % mount -o loop,offset=... image-file /mnt # use *partition* as a disk!
> % dd --sparse if=mmc-fat-part of=/mnt
> % umount /mnt
>
> And use image-file instead.

Should I prepare it for you or you can do it yourself?

--
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-14 Thread Andy Shevchenko
On Tue, Jan 14, 2020 at 1:14 AM Heinrich Schuchardt  wrote:
>
> On 1/13/20 10:52 PM, Andy Shevchenko wrote:
> > On Mon, Jan 13, 2020 at 11:05 PM Heinrich Schuchardt  
> > wrote:
> >> On 1/13/20 9:58 PM, Andy Shevchenko wrote:
> >>> On Mon, Jan 13, 2020 at 9:22 PM Andy Shevchenko
> >>>  wrote:
> >>> https://paste.cathedral-networks.org/OverdoseSegment
> >>>
> >>> I dunno how long it will be available.
> >>> I created it using
> >>> % dd if=/dev/mmcblk0p9 of=mmc-fat-part conv=sparse
> >>> % gzip mmc-fat-part
> >>>
> >> Doesn't work for me:
> >>
> >> "File Quota Exceeded"
> >>
> >> I guess Github wouldn't give you any trouble.
> >
> > https://gist.github.com/andy-shev/469aef8dfcd8f5605cb8992cf5958769
> >
>
> This image loads fine on current U-Boot, see below.

Of course it does *in the test case you have done*.
I'm describing different one. The provided image must be a *partition*
on the real disk.

So, before use it the preparatory steps must be made.

Something like

% dd if=/dev/zero of=image-file bs=1M count=1000
% fdisk image-file
 ...create a partition table, where one partition has a (similar) size
of the image I provided
% mount -o loop,offset=... image-file /mnt # use *partition* as a disk!
% dd --sparse if=mmc-fat-part of=/mnt
% umount /mnt

And use image-file instead.

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Heinrich Schuchardt

On 1/13/20 10:52 PM, Andy Shevchenko wrote:

On Mon, Jan 13, 2020 at 11:05 PM Heinrich Schuchardt  wrote:

On 1/13/20 9:58 PM, Andy Shevchenko wrote:

On Mon, Jan 13, 2020 at 9:22 PM Andy Shevchenko
 wrote:
https://paste.cathedral-networks.org/OverdoseSegment

I dunno how long it will be available.
I created it using
% dd if=/dev/mmcblk0p9 of=mmc-fat-part conv=sparse
% gzip mmc-fat-part


Doesn't work for me:

"File Quota Exceeded"

I guess Github wouldn't give you any trouble.


https://gist.github.com/andy-shev/469aef8dfcd8f5605cb8992cf5958769



This image loads fine on current U-Boot, see below.

So if you have trouble loading the same image on your machine from MMC
the problem might be with MMC and not with FAT.

Best regards

Heinrich

make qemu_arm64_defconfig
export CROSS_COMPILE=aarch64-linux-gnu-
make

qemu-system-aarch64 -machine virt -m 1G -smp cores=2 \
-bios u-boot.bin -cpu cortex-a53 -nographic -gdb tcp::1234 \
-netdev user,id=eth0,tftp=tftp -device e1000,netdev=eth0 \
-drive if=none,file=foo.img,format=raw,id=mydisk \
-device virtio-rng-pci \
-device ich9-ahci,id=ahci -device ide-drive,drive=mydisk,bus=ahci.0

U-Boot 2020.01-00268-ge936504885-dirty (Jan 12 2020 - 20:27:36 +0100)

DRAM:  1 GiB
Flash: 128 MiB
*** Warning - bad CRC, using default environment

In:pl011@900
Out:   pl011@900
Err:   pl011@900
Net:   No ethernet found.
Hit any key to stop autoboot:  0
=> scsi scan
scanning bus for devices...
Target spinup took 0 ms.
SATA link 1 timeout.
SATA link 2 timeout.
SATA link 3 timeout.
SATA link 4 timeout.
SATA link 5 timeout.
AHCI 0001. 32 slots 6 ports 1.5 Gbps 0x3f impl SATA mode
flags: 64bit ncq only
  Device 0: (0:0) Vendor: ATA Prod.: QEMU HARDDISK Rev: 2.5+
Type: Hard Disk
Capacity: 768.0 MB = 0.7 GB (1572864 x 512)
=> ls scsi 0:1
System Volume Information/
   23   New Text Document.txt
$RECYCLE.BIN/

1 file(s), 2 dir(s)

=> load scsi 0:1 0x4020 'New Text Document.txt'
23 bytes read in 11 ms (2 KiB/s)
=> md.b 4020 20
4020: 46 69 6c 65 20 6f 6e 20 46 41 54 33 32 20 70 61
  File on FAT32 pa
40200010: 72 74 69 74 69 6f 6e 00 00 00 00 00 00 00 00 00
  rtition.
=>


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Andy Shevchenko
On Mon, Jan 13, 2020 at 11:05 PM Heinrich Schuchardt  wrote:
> On 1/13/20 9:58 PM, Andy Shevchenko wrote:
> > On Mon, Jan 13, 2020 at 9:22 PM Andy Shevchenko
> >  wrote:
> > https://paste.cathedral-networks.org/OverdoseSegment
> >
> > I dunno how long it will be available.
> > I created it using
> > % dd if=/dev/mmcblk0p9 of=mmc-fat-part conv=sparse
> > % gzip mmc-fat-part
> >
> Doesn't work for me:
>
> "File Quota Exceeded"
>
> I guess Github wouldn't give you any trouble.

https://gist.github.com/andy-shev/469aef8dfcd8f5605cb8992cf5958769

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Heinrich Schuchardt

On 1/13/20 9:58 PM, Andy Shevchenko wrote:

On Mon, Jan 13, 2020 at 9:22 PM Andy Shevchenko
 wrote:

On Mon, Jan 13, 2020 at 9:15 PM Andy Shevchenko
 wrote:



P.S. I'll prepare a compressed / sparse image later as promised.


https://paste.cathedral-networks.org/OverdoseSegment

I dunno how long it will be available.
I created it using
% dd if=/dev/mmcblk0p9 of=mmc-fat-part conv=sparse
% gzip mmc-fat-part


Doesn't work for me:

"File Quota Exceeded"

I guess Github wouldn't give you any trouble.

Best regards

Heinrich


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Andy Shevchenko
On Mon, Jan 13, 2020 at 9:22 PM Andy Shevchenko
 wrote:
> On Mon, Jan 13, 2020 at 9:15 PM Andy Shevchenko
>  wrote:

> P.S. I'll prepare a compressed / sparse image later as promised.

https://paste.cathedral-networks.org/OverdoseSegment

I dunno how long it will be available.
I created it using
% dd if=/dev/mmcblk0p9 of=mmc-fat-part conv=sparse
% gzip mmc-fat-part

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Andy Shevchenko
On Mon, Jan 13, 2020 at 9:15 PM Andy Shevchenko
 wrote:
> On Mon, Jan 13, 2020 at 7:55 PM Heinrich Schuchardt  
> wrote:
> > On 1/13/20 5:34 PM, Tom Rini wrote:
> > > On Mon, Jan 13, 2020 at 12:52:48PM +0200, Andy Shevchenko wrote:

...

> > But for me the U-Boot's load command loaded a file created on Windows
> > without problems. The size of my FAT partition was 50 MiB.

I think you would like to know the size of mine. It's about 760MB.

P.S. I'll prepare a compressed / sparse image later as promised.

-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Andy Shevchenko
On Mon, Jan 13, 2020 at 7:55 PM Heinrich Schuchardt  wrote:
> On 1/13/20 5:34 PM, Tom Rini wrote:
> > On Mon, Jan 13, 2020 at 12:52:48PM +0200, Andy Shevchenko wrote:
> >> Hi!
> >>
> >> I recently stumble over FAT partitioning issue. I have a device with
> >> MMC, one partition of which has been exported as disk via USB Mass
> >> Storage to the host. When Windows 10 sees that disk it can't handle it
> >> till I format it there.
>
> This I could reproduce.
>
> So far so good. However, the same partition
> >> can't be used under U-Boot due to Windows flow, i.e. it makes a
> >> partitioning on top of the actual partition while U-Boot expects that
> >> we only have one MBR (or partitioning) on the dist. So, the commands
> >> such fatls, fatload do not recognise any file there.
>
> But for me the U-Boot's load command loaded a file created on Windows
> without problems. The size of my FAT partition was 50 MiB.
>
> Please, provide a more detailed instruction how to create an image that
> is recognized by Windows but not by U-Boot.

Windows GUI (Disk manager by clicking right button on the Windows icon
on the bar) provides this.
https://www.windowscentral.com/how-format-usb-flash-drive-windows-10

Basically I followed above (Quick or not format makes no difference --
it has the "new simple volume" available only which creates, as
mentioned, volume with all structures).

> Please, provide a gzipped test image for download.

What image?
Disk one? Let me prepare it later when I come home.

>
> Best regards
>
> Heinrich
>
> >>
> >> In Linux it's easy to use: mount -o loop,offset=65536 /dev/mmcblk0p9 /mnt
> >> (offset can be different)
> >>
> >> But I would like to use it in U-Boot.
> >>
> >> Any thoughts on this?
> >
> > I'm not sure.  If you didn't have to use offset+loop to re-expose things
> > as a new disk and show the partition I'd say we need to support that
> > case.  But here, hmmm.  We don't have "loopback".  So, I'm not sure.
> > Can't you just tell Windows to use the whole device as FAT and so
> > mmcblk0p9 would directly be FAT and work as expected in both Linux and
> > U-Boot?
> >



-- 
With Best Regards,
Andy Shevchenko


Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Heinrich Schuchardt

On 1/13/20 5:34 PM, Tom Rini wrote:

On Mon, Jan 13, 2020 at 12:52:48PM +0200, Andy Shevchenko wrote:

Hi!

I recently stumble over FAT partitioning issue. I have a device with
MMC, one partition of which has been exported as disk via USB Mass
Storage to the host. When Windows 10 sees that disk it can't handle it
till I format it there.


This I could reproduce.

So far so good. However, the same partition

can't be used under U-Boot due to Windows flow, i.e. it makes a
partitioning on top of the actual partition while U-Boot expects that
we only have one MBR (or partitioning) on the dist. So, the commands
such fatls, fatload do not recognise any file there.


But for me the U-Boot's load command loaded a file created on Windows
without problems. The size of my FAT partition was 50 MiB.

Please, provide a more detailed instruction how to create an image that
is recognized by Windows but not by U-Boot.

Please, provide a gzipped test image for download.

Best regards

Heinrich



In Linux it's easy to use: mount -o loop,offset=65536 /dev/mmcblk0p9 /mnt
(offset can be different)

But I would like to use it in U-Boot.

Any thoughts on this?


I'm not sure.  If you didn't have to use offset+loop to re-expose things
as a new disk and show the partition I'd say we need to support that
case.  But here, hmmm.  We don't have "loopback".  So, I'm not sure.
Can't you just tell Windows to use the whole device as FAT and so
mmcblk0p9 would directly be FAT and work as expected in both Linux and
U-Boot?



Re: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Tom Rini
On Mon, Jan 13, 2020 at 12:52:48PM +0200, Andy Shevchenko wrote:
> Hi!
> 
> I recently stumble over FAT partitioning issue. I have a device with
> MMC, one partition of which has been exported as disk via USB Mass
> Storage to the host. When Windows 10 sees that disk it can't handle it
> till I format it there. So far so good. However, the same partition
> can't be used under U-Boot due to Windows flow, i.e. it makes a
> partitioning on top of the actual partition while U-Boot expects that
> we only have one MBR (or partitioning) on the dist. So, the commands
> such fatls, fatload do not recognise any file there.
> 
> In Linux it's easy to use: mount -o loop,offset=65536 /dev/mmcblk0p9 /mnt
> (offset can be different)
> 
> But I would like to use it in U-Boot.
> 
> Any thoughts on this?

I'm not sure.  If you didn't have to use offset+loop to re-expose things
as a new disk and show the partition I'd say we need to support that
case.  But here, hmmm.  We don't have "loopback".  So, I'm not sure.
Can't you just tell Windows to use the whole device as FAT and so
mmcblk0p9 would directly be FAT and work as expected in both Linux and
U-Boot?

-- 
Tom


signature.asc
Description: PGP signature


Fwd: fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Andy Shevchenko
-- Forwarded message -
From: Andy Shevchenko 
Date: Mon, Jan 13, 2020 at 12:52 PM
Subject: fat: handle Windows formatted partition (thru USB Mass Storage)
To: AKASHI Takahiro , Heinrich Schuchardt

Cc: Tom Rini , U-Boot Mailing List
, Simon Glass , Bin Meng



Hi!

I recently stumble over FAT partitioning issue. I have a device with
MMC, one partition of which has been exported as disk via USB Mass
Storage to the host. When Windows 10 sees that disk it can't handle it
till I format it there. So far so good. However, the same partition
can't be used under U-Boot due to Windows flow, i.e. it makes a
partitioning on top of the actual partition while U-Boot expects that
we only have one MBR (or partitioning) on the dist. So, the commands
such fatls, fatload do not recognise any file there.

In Linux it's easy to use: mount -o loop,offset=65536 /dev/mmcblk0p9 /mnt
(offset can be different)

But I would like to use it in U-Boot.

Any thoughts on this?

-- 
With Best Regards,
Andy Shevchenko


fat: handle Windows formatted partition (thru USB Mass Storage)

2020-01-13 Thread Andy Shevchenko
Hi!

I recently stumble over FAT partitioning issue. I have a device with
MMC, one partition of which has been exported as disk via USB Mass
Storage to the host. When Windows 10 sees that disk it can't handle it
till I format it there. So far so good. However, the same partition
can't be used under U-Boot due to Windows flow, i.e. it makes a
partitioning on top of the actual partition while U-Boot expects that
we only have one MBR (or partitioning) on the dist. So, the commands
such fatls, fatload do not recognise any file there.

In Linux it's easy to use: mount -o loop,offset=65536 /dev/mmcblk0p9 /mnt
(offset can be different)

But I would like to use it in U-Boot.

Any thoughts on this?

-- 
With Best Regards,
Andy Shevchenko
-BEGIN PGP SIGNATURE-

iQIzBAABCgAdFiEE6HLbQJwaaH776GM2h/n2NdMddlIFAl3pgB8ACgkQh/n2NdMd
dlIamxAAlMfipD4slHtXthbyHARbN7TY7BDLUv5/zGA1NJWIlqjQHvZ5LPPUPAuu
fWLwAy2NcqcoaJD8Wwf02fnDAbJGykw0aL7FiJ2sUBHqjmKf6CmTtLT5TrvpqEBF
CpwUooxGx0BEM/BnRhVP6dPD0yXZCvz54PQlBo6qdKU+V/x4M5vEBGzdTU9gIJQd
GzkANSjkO81azav5prUMQMslDSFNi2oXgDKMrV4DJuw2g19ueTxld9BJcfswiDhP
ekow18V7Zsv2cfP37UhcTNUWliQ/6RJCXwxoCiNOmG96AXPC1eHVILowticDznAc
ivqCbmqDWgXzC5TFmsAXR9DRPSvvOWTJEZD+yltLIIFrbkmdW1dyDF18v4mTYP9G
zyuFaVHr9ZIZYxKEJFRtVB7pwBdhj+YiL/iWSfynCGQ4HyL4BbmtLyyoXLBMZrfi
fKwPJH2HKZpJ0v3kLEbNU3lrZdugjtsJOY/quQDs5g+3n1KA9UrmNm1J5Lrb1FTz
jZOd64viYhbDkMKWvSgcwdapCUI+BoyrgPwVfDFPfzIoQCWedbp/v8y/7eql3yjt
O2T51k033/QUXiUEdQ97H6klbiehje3RurMb0JNNM4/ETVp6Yk/CBroMsA9mhYV5
GadsZ6biZaJJQXEg5mlfcIdF+LBHJluZEgswfBwnV1p82D3soeM=
=USnQ
-END PGP SIGNATURE-